From: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
To: Song Yoong Siang <yoong.siang.song@intel.com>
Cc: "David S . Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@google.com>,
Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
Simon Horman <horms@kernel.org>,
Willem de Bruijn <willemb@google.com>,
Florian Bezdeka <florian.bezdeka@siemens.com>,
Donald Hunter <donald.hunter@gmail.com>,
Jonathan Corbet <corbet@lwn.net>, Bjorn Topel <bjorn@kernel.org>,
Magnus Karlsson <magnus.karlsson@intel.com>,
Jonathan Lemon <jonathan.lemon@gmail.com>,
Andrew Lunn <andrew+netdev@lunn.ch>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Jesper Dangaard Brouer <hawk@kernel.org>,
John Fastabend <john.fastabend@gmail.com>,
Joe Damato <jdamato@fastly.com>,
Stanislav Fomichev <sdf@fomichev.me>,
Xuan Zhuo <xuanzhuo@linux.alibaba.com>,
Mina Almasry <almasrymina@google.com>,
Daniel Jurgens <danielj@nvidia.com>,
Andrii Nakryiko <andrii@kernel.org>,
Eduard Zingerman <eddyz87@gmail.com>,
Mykola Lysenko <mykolal@fb.com>,
Martin KaFai Lau <martin.lau@linux.dev>,
Song Liu <song@kernel.org>,
Yonghong Song <yonghong.song@linux.dev>,
KP Singh <kpsingh@kernel.org>, Hao Luo <haoluo@google.com>,
Jiri Olsa <jolsa@kernel.org>, Shuah Khan <shuah@kernel.org>,
Alexandre Torgue <alexandre.torgue@foss.st.com>,
Jose Abreu <joabreu@synopsys.com>,
Maxime Coquelin <mcoquelin.stm32@gmail.com>,
Tony Nguyen <anthony.l.nguyen@intel.com>,
Przemek Kitszel <przemyslaw.kitszel@intel.com>,
Faizal Rahim <faizal.abdul.rahim@linux.intel.com>,
Choong Yong Liang <yong.liang.choong@linux.intel.com>,
Bouska Zdenek <zdenek.bouska@siemens.com>,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-doc@vger.kernel.org, bpf@vger.kernel.org,
linux-kselftest@vger.kernel.org,
linux-stm32@st-md-mailman.stormreply.com,
linux-arm-kernel@lists.infradead.org,
intel-wired-lan@lists.osuosl.org, xdp-hints@xdp-project.net
Subject: [xdp-hints] Re: [PATCH bpf-next v9 5/5] igc: Add launch time support to XDP ZC
Date: Thu, 6 Feb 2025 20:29:59 +0100 [thread overview]
Message-ID: <Z6UNt2GatiYYdZIW@boxer> (raw)
In-Reply-To: <20250206060408.808325-6-yoong.siang.song@intel.com>
On Thu, Feb 06, 2025 at 02:04:08PM +0800, Song Yoong Siang wrote:
> Enable Launch Time Control (LTC) support for XDP zero copy via XDP Tx
> metadata framework.
>
> This patch has been tested with tools/testing/selftests/bpf/xdp_hw_metadata
> on Intel I225-LM Ethernet controller. Below are the test steps and result.
>
> Test 1: Send a single packet with the launch time set to 1 s in the future.
>
> Test steps:
> 1. On the DUT, start the xdp_hw_metadata selftest application:
> $ sudo ./xdp_hw_metadata enp2s0 -l 1000000000 -L 1
>
> 2. On the Link Partner, send a UDP packet with VLAN priority 1 to port 9091
> of the DUT.
>
> Result:
> When the launch time is set to 1 s in the future, the delta between the
> launch time and the transmit hardware timestamp is 0.016 us, as shown in
> printout of the xdp_hw_metadata application below.
> 0x562ff5dc8880: rx_desc[4]->addr=84110 addr=84110 comp_addr=84110 EoP
> rx_hash: 0xE343384 with RSS type:0x1
> HW RX-time: 1734578015467548904 (sec:1734578015.4675)
> delta to User RX-time sec:0.0002 (183.103 usec)
> XDP RX-time: 1734578015467651698 (sec:1734578015.4677)
> delta to User RX-time sec:0.0001 (80.309 usec)
> No rx_vlan_tci or rx_vlan_proto, err=-95
> 0x562ff5dc8880: ping-pong with csum=561c (want c7dd)
> csum_start=34 csum_offset=6
> HW RX-time: 1734578015467548904 (sec:1734578015.4675)
> delta to HW Launch-time sec:1.0000 (1000000.000 usec)
> 0x562ff5dc8880: complete tx idx=4 addr=4018
> HW Launch-time: 1734578016467548904 (sec:1734578016.4675)
> delta to HW TX-complete-time sec:0.0000 (0.016 usec)
> HW TX-complete-time: 1734578016467548920 (sec:1734578016.4675)
> delta to User TX-complete-time sec:0.0000
> (32.546 usec)
> XDP RX-time: 1734578015467651698 (sec:1734578015.4677)
> delta to User TX-complete-time sec:0.9999
> (999929.768 usec)
> HW RX-time: 1734578015467548904 (sec:1734578015.4675)
> delta to HW TX-complete-time sec:1.0000 (1000000.016 usec)
> 0x562ff5dc8880: complete rx idx=132 addr=84110
>
> Test 2: Send 1000 packets with a 10 ms interval and the launch time set to
> 500 us in the future.
>
> Test steps:
> 1. On the DUT, start the xdp_hw_metadata selftest application:
> $ sudo chrt -f 99 ./xdp_hw_metadata enp2s0 -l 500000 -L 1 > \
> /dev/shm/result.log
>
> 2. On the Link Partner, send 1000 UDP packets with a 10 ms interval and
> VLAN priority 1 to port 9091 of the DUT.
>
> Result:
> When the launch time is set to 500 us in the future, the average delta
> between the launch time and the transmit hardware timestamp is 0.016 us,
> as shown in the analysis of /dev/shm/result.log below. The XDP launch time
> works correctly in sending 1000 packets continuously.
> Min delta: 0.005 us
> Avr delta: 0.016 us
> Max delta: 0.031 us
> Total packets forwarded: 1000
>
> Signed-off-by: Song Yoong Siang <yoong.siang.song@intel.com>
Reviewed-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
also with one nit. Thanks!
> ---
> drivers/net/ethernet/intel/igc/igc.h | 1 +
> drivers/net/ethernet/intel/igc/igc_main.c | 57 ++++++++++++++++++++++-
> 2 files changed, 56 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/igc/igc.h b/drivers/net/ethernet/intel/igc/igc.h
> index b8111ad9a9a8..cd1d7b6c1782 100644
> --- a/drivers/net/ethernet/intel/igc/igc.h
> +++ b/drivers/net/ethernet/intel/igc/igc.h
> @@ -579,6 +579,7 @@ struct igc_metadata_request {
> struct xsk_tx_metadata *meta;
> struct igc_ring *tx_ring;
> u32 cmd_type;
> + u16 used_desc;
> };
>
> struct igc_q_vector {
> diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
> index 3df608601a4b..f239f744247d 100644
> --- a/drivers/net/ethernet/intel/igc/igc_main.c
> +++ b/drivers/net/ethernet/intel/igc/igc_main.c
> @@ -2973,9 +2973,44 @@ static u64 igc_xsk_fill_timestamp(void *_priv)
> return *(u64 *)_priv;
> }
>
> +static void igc_xsk_request_launch_time(u64 launch_time, void *_priv)
> +{
> + struct igc_metadata_request *meta_req = _priv;
> + struct igc_ring *tx_ring = meta_req->tx_ring;
> + __le32 launch_time_offset;
> + bool insert_empty = false;
> + bool first_flag = false;
> +
> + if (!tx_ring->launchtime_enable)
> + return;
> +
> + launch_time_offset = igc_tx_launchtime(tx_ring,
> + ns_to_ktime(launch_time),
> + &first_flag, &insert_empty);
> + if (insert_empty) {
> + /* Disregard the launch time request if the required empty frame
> + * fails to be inserted.
> + */
> + if (igc_insert_empty_frame(tx_ring))
> + return;
> +
> + meta_req->tx_buffer =
> + &tx_ring->tx_buffer_info[tx_ring->next_to_use];
> + /* Inserting an empty packet requires two descriptors:
> + * one data descriptor and one context descriptor.
> + */
> + meta_req->used_desc += 2;
tiny nit: you could accumulate the amount of used descs in local variable
and update the igc_metadata_request::used_desc once.
> + }
> +
> + /* Use one context descriptor to specify launch time and first flag. */
> + igc_tx_ctxtdesc(tx_ring, launch_time_offset, first_flag, 0, 0, 0);
> + meta_req->used_desc += 1;
> +}
> +
> const struct xsk_tx_metadata_ops igc_xsk_tx_metadata_ops = {
> .tmo_request_timestamp = igc_xsk_request_timestamp,
> .tmo_fill_timestamp = igc_xsk_fill_timestamp,
> + .tmo_request_launch_time = igc_xsk_request_launch_time,
> };
>
> static void igc_xdp_xmit_zc(struct igc_ring *ring)
> @@ -2998,7 +3033,13 @@ static void igc_xdp_xmit_zc(struct igc_ring *ring)
> ntu = ring->next_to_use;
> budget = igc_desc_unused(ring);
>
> - while (xsk_tx_peek_desc(pool, &xdp_desc) && budget--) {
> + /* Packets with launch time require one data descriptor and one context
> + * descriptor. When the launch time falls into the next Qbv cycle, we
> + * may need to insert an empty packet, which requires two more
> + * descriptors. Therefore, to be safe, we always ensure we have at least
> + * 4 descriptors available.
> + */
> + while (xsk_tx_peek_desc(pool, &xdp_desc) && budget >= 4) {
> struct igc_metadata_request meta_req;
> struct xsk_tx_metadata *meta = NULL;
> struct igc_tx_buffer *bi;
> @@ -3019,9 +3060,19 @@ static void igc_xdp_xmit_zc(struct igc_ring *ring)
> meta_req.tx_ring = ring;
> meta_req.tx_buffer = bi;
> meta_req.meta = meta;
> + meta_req.used_desc = 0;
> xsk_tx_metadata_request(meta, &igc_xsk_tx_metadata_ops,
> &meta_req);
>
> + /* xsk_tx_metadata_request() may have updated next_to_use */
> + ntu = ring->next_to_use;
> +
> + /* xsk_tx_metadata_request() may have updated Tx buffer info */
> + bi = meta_req.tx_buffer;
> +
> + /* xsk_tx_metadata_request() may use a few descriptors */
> + budget -= meta_req.used_desc;
> +
> tx_desc = IGC_TX_DESC(ring, ntu);
> tx_desc->read.cmd_type_len = cpu_to_le32(meta_req.cmd_type);
> tx_desc->read.olinfo_status = cpu_to_le32(olinfo_status);
> @@ -3039,9 +3090,11 @@ static void igc_xdp_xmit_zc(struct igc_ring *ring)
> ntu++;
> if (ntu == ring->count)
> ntu = 0;
> +
> + ring->next_to_use = ntu;
> + budget--;
> }
>
> - ring->next_to_use = ntu;
> if (tx_desc) {
> igc_flush_tx_descriptors(ring);
> xsk_tx_release(pool);
> --
> 2.34.1
>
prev parent reply other threads:[~2025-02-06 19:30 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-02-06 6:04 [xdp-hints] [PATCH bpf-next v9 0/5] xsk: TX metadata Launch Time support Song Yoong Siang
2025-02-06 6:04 ` [xdp-hints] [PATCH bpf-next v9 1/5] xsk: Add launch time hardware offload support to XDP Tx metadata Song Yoong Siang
2025-02-06 6:04 ` [xdp-hints] [PATCH bpf-next v9 2/5] selftests/bpf: Add launch time request to xdp_hw_metadata Song Yoong Siang
2025-02-06 6:04 ` [xdp-hints] [PATCH bpf-next v9 3/5] net: stmmac: Add launch time support to XDP ZC Song Yoong Siang
2025-02-06 19:08 ` [xdp-hints] " Maciej Fijalkowski
2025-02-06 6:04 ` [xdp-hints] [PATCH bpf-next v9 4/5] igc: Refactor empty frame insertion for launch time support Song Yoong Siang
2025-02-06 19:23 ` [xdp-hints] " Maciej Fijalkowski
2025-02-06 6:04 ` [xdp-hints] [PATCH bpf-next v9 5/5] igc: Add launch time support to XDP ZC Song Yoong Siang
2025-02-06 19:29 ` Maciej Fijalkowski [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://lists.xdp-project.net/postorius/lists/xdp-hints.xdp-project.net/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Z6UNt2GatiYYdZIW@boxer \
--to=maciej.fijalkowski@intel.com \
--cc=alexandre.torgue@foss.st.com \
--cc=almasrymina@google.com \
--cc=andrew+netdev@lunn.ch \
--cc=andrii@kernel.org \
--cc=anthony.l.nguyen@intel.com \
--cc=ast@kernel.org \
--cc=bjorn@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=daniel@iogearbox.net \
--cc=danielj@nvidia.com \
--cc=davem@davemloft.net \
--cc=donald.hunter@gmail.com \
--cc=eddyz87@gmail.com \
--cc=edumazet@google.com \
--cc=faizal.abdul.rahim@linux.intel.com \
--cc=florian.bezdeka@siemens.com \
--cc=haoluo@google.com \
--cc=hawk@kernel.org \
--cc=horms@kernel.org \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jdamato@fastly.com \
--cc=joabreu@synopsys.com \
--cc=john.fastabend@gmail.com \
--cc=jolsa@kernel.org \
--cc=jonathan.lemon@gmail.com \
--cc=kpsingh@kernel.org \
--cc=kuba@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=linux-stm32@st-md-mailman.stormreply.com \
--cc=magnus.karlsson@intel.com \
--cc=martin.lau@linux.dev \
--cc=mcoquelin.stm32@gmail.com \
--cc=mykolal@fb.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=przemyslaw.kitszel@intel.com \
--cc=sdf@fomichev.me \
--cc=shuah@kernel.org \
--cc=song@kernel.org \
--cc=willemb@google.com \
--cc=xdp-hints@xdp-project.net \
--cc=xuanzhuo@linux.alibaba.com \
--cc=yong.liang.choong@linux.intel.com \
--cc=yonghong.song@linux.dev \
--cc=yoong.siang.song@intel.com \
--cc=zdenek.bouska@siemens.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox