* [xdp-hints] [PATCH net v3 1/1] igc: read before write to SRRCTL register
@ 2023-04-14 15:49 Song Yoong Siang
2023-04-14 20:05 ` [xdp-hints] " Jesper Dangaard Brouer
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Song Yoong Siang @ 2023-04-14 15:49 UTC (permalink / raw)
To: Jesse Brandeburg, Tony Nguyen, David S . Miller, Eric Dumazet,
Jakub Kicinski, Paolo Abeni, Alexei Starovoitov, Daniel Borkmann,
Jesper Dangaard Brouer, John Fastabend, Maciej Fijalkowski,
Vedang Patel, Jithu Joseph, Andre Guedes, Jesper Dangaard Brouer,
Stanislav Fomichev, Jacob Keller, David Laight
Cc: intel-wired-lan, netdev, linux-kernel, bpf, xdp-hints, stable,
Song Yoong Siang
igc_configure_rx_ring() function will be called as part of XDP program
setup. If Rx hardware timestamp is enabled prio to XDP program setup,
this timestamp enablement will be overwritten when buffer size is
written into SRRCTL register.
Thus, this commit read the register value before write to SRRCTL
register. This commit is tested by using xdp_hw_metadata bpf selftest
tool. The tool enables Rx hardware timestamp and then attach XDP program
to igc driver. It will display hardware timestamp of UDP packet with
port number 9092. Below are detail of test steps and results.
Command on DUT:
sudo ./xdp_hw_metadata <interface name>
Command on Link Partner:
echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
Result before this patch:
skb hwtstamp is not found!
Result after this patch:
found skb hwtstamp = 1677800973.642836757
Optionally, read PHC to confirm the values obtained are almost the same:
Command:
sudo ./testptp -d /dev/ptp0 -g
Result:
clock time: 1677800973.913598978 or Fri Mar 3 07:49:33 2023
Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy")
Cc: <stable@vger.kernel.org> # 5.14+
Signed-off-by: Song Yoong Siang <yoong.siang.song@intel.com>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Jesper Dangaard Brouer <brouer@redhat.com>
---
v2 -> v3: Refactor SRRCTL definitions to more human readable definitions
v1 -> v2: Fix indention
---
drivers/net/ethernet/intel/igc/igc_base.h | 11 ++++++++---
drivers/net/ethernet/intel/igc/igc_main.c | 7 +++++--
2 files changed, 13 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h
index 7a992befca24..9f3827eda157 100644
--- a/drivers/net/ethernet/intel/igc/igc_base.h
+++ b/drivers/net/ethernet/intel/igc/igc_base.h
@@ -87,8 +87,13 @@ union igc_adv_rx_desc {
#define IGC_RXDCTL_SWFLUSH 0x04000000 /* Receive Software Flush */
/* SRRCTL bit definitions */
-#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
-#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
-#define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF 0x02000000
+#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0)
+#define IGC_SRRCTL_BSIZEPKT(x) FIELD_PREP(IGC_SRRCTL_BSIZEPKT_MASK, \
+ (x) / 1024) /* in 1 KB resolution */
+#define IGC_SRRCTL_BSIZEHDR_MASK GENMASK(13, 8)
+#define IGC_SRRCTL_BSIZEHDR(x) FIELD_PREP(IGC_SRRCTL_BSIZEHDR_MASK, \
+ (x) / 64) /* in 64 bytes resolution */
+#define IGC_SRRCTL_DESCTYPE_MASK GENMASK(27, 25)
+#define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF FIELD_PREP(IGC_SRRCTL_DESCTYPE_MASK, 1)
#endif /* _IGC_BASE_H */
diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
index 25fc6c65209b..a2d823e64609 100644
--- a/drivers/net/ethernet/intel/igc/igc_main.c
+++ b/drivers/net/ethernet/intel/igc/igc_main.c
@@ -641,8 +641,11 @@ static void igc_configure_rx_ring(struct igc_adapter *adapter,
else
buf_size = IGC_RXBUFFER_2048;
- srrctl = IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
- srrctl |= buf_size >> IGC_SRRCTL_BSIZEPKT_SHIFT;
+ srrctl = rd32(IGC_SRRCTL(reg_idx));
+ srrctl &= ~(IGC_SRRCTL_BSIZEPKT_MASK | IGC_SRRCTL_BSIZEHDR_MASK |
+ IGC_SRRCTL_DESCTYPE_MASK);
+ srrctl |= IGC_SRRCTL_BSIZEHDR(IGC_RX_HDR_LEN);
+ srrctl |= IGC_SRRCTL_BSIZEPKT(buf_size);
srrctl |= IGC_SRRCTL_DESCTYPE_ADV_ONEBUF;
wr32(IGC_SRRCTL(reg_idx), srrctl);
--
2.34.1
^ permalink raw reply [flat|nested] 9+ messages in thread* [xdp-hints] Re: [PATCH net v3 1/1] igc: read before write to SRRCTL register
2023-04-14 15:49 [xdp-hints] [PATCH net v3 1/1] igc: read before write to SRRCTL register Song Yoong Siang
@ 2023-04-14 20:05 ` Jesper Dangaard Brouer
2023-04-17 14:24 ` Jesper Dangaard Brouer
2023-04-15 9:19 ` Florian Bezdeka
2023-04-30 5:05 ` [xdp-hints] Re: [Intel-wired-lan] " naamax.meir
2 siblings, 1 reply; 9+ messages in thread
From: Jesper Dangaard Brouer @ 2023-04-14 20:05 UTC (permalink / raw)
To: Song Yoong Siang, Jesse Brandeburg, Tony Nguyen,
David S . Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Alexei Starovoitov, Daniel Borkmann, Jesper Dangaard Brouer,
John Fastabend, Maciej Fijalkowski, Vedang Patel, Jithu Joseph,
Andre Guedes, Stanislav Fomichev, Jacob Keller, David Laight
Cc: brouer, intel-wired-lan, netdev, linux-kernel, bpf, xdp-hints, stable
On 14/04/2023 17.49, Song Yoong Siang wrote:
> igc_configure_rx_ring() function will be called as part of XDP program
> setup. If Rx hardware timestamp is enabled prio to XDP program setup,
> this timestamp enablement will be overwritten when buffer size is
> written into SRRCTL register.
>
> Thus, this commit read the register value before write to SRRCTL
> register. This commit is tested by using xdp_hw_metadata bpf selftest
> tool. The tool enables Rx hardware timestamp and then attach XDP program
> to igc driver. It will display hardware timestamp of UDP packet with
> port number 9092. Below are detail of test steps and results.
>
> Command on DUT:
> sudo ./xdp_hw_metadata <interface name>
>
> Command on Link Partner:
> echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
>
> Result before this patch:
> skb hwtstamp is not found!
>
> Result after this patch:
> found skb hwtstamp = 1677800973.642836757
>
> Optionally, read PHC to confirm the values obtained are almost the same:
> Command:
> sudo ./testptp -d /dev/ptp0 -g
> Result:
> clock time: 1677800973.913598978 or Fri Mar 3 07:49:33 2023
>
> Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy")
> Cc: <stable@vger.kernel.org> # 5.14+
> Signed-off-by: Song Yoong Siang <yoong.siang.song@intel.com>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Jesper Dangaard Brouer <brouer@redhat.com>
> ---
LGTM, thank for the adjustments :-)
Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
> v2 -> v3: Refactor SRRCTL definitions to more human readable definitions
> v1 -> v2: Fix indention
> ---
> drivers/net/ethernet/intel/igc/igc_base.h | 11 ++++++++---
> drivers/net/ethernet/intel/igc/igc_main.c | 7 +++++--
> 2 files changed, 13 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h
> index 7a992befca24..9f3827eda157 100644
> --- a/drivers/net/ethernet/intel/igc/igc_base.h
> +++ b/drivers/net/ethernet/intel/igc/igc_base.h
> @@ -87,8 +87,13 @@ union igc_adv_rx_desc {
> #define IGC_RXDCTL_SWFLUSH 0x04000000 /* Receive Software Flush */
>
> /* SRRCTL bit definitions */
> -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */
> -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */
> -#define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF 0x02000000
> +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0)
> +#define IGC_SRRCTL_BSIZEPKT(x) FIELD_PREP(IGC_SRRCTL_BSIZEPKT_MASK, \
> + (x) / 1024) /* in 1 KB resolution */
> +#define IGC_SRRCTL_BSIZEHDR_MASK GENMASK(13, 8)
> +#define IGC_SRRCTL_BSIZEHDR(x) FIELD_PREP(IGC_SRRCTL_BSIZEHDR_MASK, \
> + (x) / 64) /* in 64 bytes resolution */
> +#define IGC_SRRCTL_DESCTYPE_MASK GENMASK(27, 25)
> +#define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF FIELD_PREP(IGC_SRRCTL_DESCTYPE_MASK, 1)
>
> #endif /* _IGC_BASE_H */
> diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
> index 25fc6c65209b..a2d823e64609 100644
> --- a/drivers/net/ethernet/intel/igc/igc_main.c
> +++ b/drivers/net/ethernet/intel/igc/igc_main.c
> @@ -641,8 +641,11 @@ static void igc_configure_rx_ring(struct igc_adapter *adapter,
> else
> buf_size = IGC_RXBUFFER_2048;
>
> - srrctl = IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
> - srrctl |= buf_size >> IGC_SRRCTL_BSIZEPKT_SHIFT;
> + srrctl = rd32(IGC_SRRCTL(reg_idx));
> + srrctl &= ~(IGC_SRRCTL_BSIZEPKT_MASK | IGC_SRRCTL_BSIZEHDR_MASK |
> + IGC_SRRCTL_DESCTYPE_MASK);
> + srrctl |= IGC_SRRCTL_BSIZEHDR(IGC_RX_HDR_LEN);
> + srrctl |= IGC_SRRCTL_BSIZEPKT(buf_size);
> srrctl |= IGC_SRRCTL_DESCTYPE_ADV_ONEBUF;
>
> wr32(IGC_SRRCTL(reg_idx), srrctl);
^ permalink raw reply [flat|nested] 9+ messages in thread* [xdp-hints] Re: [PATCH net v3 1/1] igc: read before write to SRRCTL register
2023-04-14 20:05 ` [xdp-hints] " Jesper Dangaard Brouer
@ 2023-04-17 14:24 ` Jesper Dangaard Brouer
2023-04-18 9:31 ` Florian Bezdeka
0 siblings, 1 reply; 9+ messages in thread
From: Jesper Dangaard Brouer @ 2023-04-17 14:24 UTC (permalink / raw)
To: Song Yoong Siang, Bezdeka, Florian, Jesse Brandeburg,
Tony Nguyen, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Alexei Starovoitov, Daniel Borkmann,
Jesper Dangaard Brouer, John Fastabend, Maciej Fijalkowski,
Vedang Patel, Jithu Joseph, Andre Guedes, Stanislav Fomichev,
Jacob Keller, David Laight
Cc: brouer, intel-wired-lan, netdev, linux-kernel, bpf, xdp-hints, stable
On 14/04/2023 22.05, Jesper Dangaard Brouer wrote:
>
> On 14/04/2023 17.49, Song Yoong Siang wrote:
>> igc_configure_rx_ring() function will be called as part of XDP program
>> setup. If Rx hardware timestamp is enabled prio to XDP program setup,
>> this timestamp enablement will be overwritten when buffer size is
>> written into SRRCTL register.
>>
>> Thus, this commit read the register value before write to SRRCTL
>> register. This commit is tested by using xdp_hw_metadata bpf selftest
>> tool. The tool enables Rx hardware timestamp and then attach XDP program
>> to igc driver. It will display hardware timestamp of UDP packet with
>> port number 9092. Below are detail of test steps and results.
>>
[...]
>>
>> Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy")
>> Cc: <stable@vger.kernel.org> # 5.14+
>> Signed-off-by: Song Yoong Siang <yoong.siang.song@intel.com>
>> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
>> Reviewed-by: Jesper Dangaard Brouer <brouer@redhat.com>
>> ---
>
> LGTM, thank for the adjustments :-)
>
> Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
>
Tested-by: Jesper Dangaard Brouer <brouer@redhat.com>
I can confirm that this patch fix the issue I experienced with igc.
This patch clearly fixes a bug in igc when writing the SRRCTL register.
(as bit 30 in register is "Timestamp Received Packet" which got cleared
before).
Florian might have found another bug around RX timestamps, but this
patch should be safe and sane to apply as is.
>> v2 -> v3: Refactor SRRCTL definitions to more human readable definitions
>> v1 -> v2: Fix indention
>> ---
>> drivers/net/ethernet/intel/igc/igc_base.h | 11 ++++++++---
>> drivers/net/ethernet/intel/igc/igc_main.c | 7 +++++--
>> 2 files changed, 13 insertions(+), 5 deletions(-)
^ permalink raw reply [flat|nested] 9+ messages in thread* [xdp-hints] Re: [PATCH net v3 1/1] igc: read before write to SRRCTL register
2023-04-17 14:24 ` Jesper Dangaard Brouer
@ 2023-04-18 9:31 ` Florian Bezdeka
0 siblings, 0 replies; 9+ messages in thread
From: Florian Bezdeka @ 2023-04-18 9:31 UTC (permalink / raw)
To: Jesper Dangaard Brouer, Song Yoong Siang, Jesse Brandeburg,
Tony Nguyen, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Alexei Starovoitov, Daniel Borkmann,
Jesper Dangaard Brouer, John Fastabend, Maciej Fijalkowski,
Vedang Patel, Jithu Joseph, Andre Guedes, Stanislav Fomichev,
Jacob Keller, David Laight
Cc: brouer, intel-wired-lan, netdev, linux-kernel, bpf, xdp-hints, stable
On Mon, 2023-04-17 at 16:24 +0200, Jesper Dangaard Brouer wrote:
> On 14/04/2023 22.05, Jesper Dangaard Brouer wrote:
> >
> > On 14/04/2023 17.49, Song Yoong Siang wrote:
> > > igc_configure_rx_ring() function will be called as part of XDP program
> > > setup. If Rx hardware timestamp is enabled prio to XDP program setup,
> > > this timestamp enablement will be overwritten when buffer size is
> > > written into SRRCTL register.
> > >
> > > Thus, this commit read the register value before write to SRRCTL
> > > register. This commit is tested by using xdp_hw_metadata bpf selftest
> > > tool. The tool enables Rx hardware timestamp and then attach XDP program
> > > to igc driver. It will display hardware timestamp of UDP packet with
> > > port number 9092. Below are detail of test steps and results.
> > >
> [...]
> > >
> > > Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy")
> > > Cc: <stable@vger.kernel.org> # 5.14+
> > > Signed-off-by: Song Yoong Siang <yoong.siang.song@intel.com>
> > > Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> > > Reviewed-by: Jesper Dangaard Brouer <brouer@redhat.com>
> > > ---
> >
> > LGTM, thank for the adjustments :-)
> >
> > Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
> >
>
> Tested-by: Jesper Dangaard Brouer <brouer@redhat.com>
>
> I can confirm that this patch fix the issue I experienced with igc.
>
> This patch clearly fixes a bug in igc when writing the SRRCTL register.
> (as bit 30 in register is "Timestamp Received Packet" which got cleared
> before).
>
> Florian might have found another bug around RX timestamps, but this
> patch should be safe and sane to apply as is.
After a closer look I'm quite sure now that this patch should fix my
issue as well. The register will be overwritten when setting up a
XSK_POOL as well:
igc_bpf
igc_xdp_setup_pool
igc_enable_rx_ring
igc_configure_rx_ring
wr32(IGC_SRRCTL)
I already removed the BPF loading (which is the use case that the patch
description mentions) from my setup to limit the search scope. If you
like you could extend the patch description, but I'm fine with it.
Thanks a lot for all the support / ideas! Highly appreciated!
Florian
>
> > > v2 -> v3: Refactor SRRCTL definitions to more human readable definitions
> > > v1 -> v2: Fix indention
> > > ---
> > > drivers/net/ethernet/intel/igc/igc_base.h | 11 ++++++++---
> > > drivers/net/ethernet/intel/igc/igc_main.c | 7 +++++--
> > > 2 files changed, 13 insertions(+), 5 deletions(-)
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* [xdp-hints] Re: [PATCH net v3 1/1] igc: read before write to SRRCTL register
2023-04-14 15:49 [xdp-hints] [PATCH net v3 1/1] igc: read before write to SRRCTL register Song Yoong Siang
2023-04-14 20:05 ` [xdp-hints] " Jesper Dangaard Brouer
@ 2023-04-15 9:19 ` Florian Bezdeka
2023-04-16 2:19 ` Song, Yoong Siang
2023-04-30 5:05 ` [xdp-hints] Re: [Intel-wired-lan] " naamax.meir
2 siblings, 1 reply; 9+ messages in thread
From: Florian Bezdeka @ 2023-04-15 9:19 UTC (permalink / raw)
To: Song Yoong Siang, Jesse Brandeburg, Tony Nguyen,
David S . Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Alexei Starovoitov, Daniel Borkmann, Jesper Dangaard Brouer,
John Fastabend, Maciej Fijalkowski, Vedang Patel, Jithu Joseph,
Andre Guedes, Jesper Dangaard Brouer, Stanislav Fomichev,
Jacob Keller, David Laight
Cc: intel-wired-lan, netdev, linux-kernel, bpf, xdp-hints, stable
On 14.04.23 17:49, Song Yoong Siang wrote:
> igc_configure_rx_ring() function will be called as part of XDP program
> setup. If Rx hardware timestamp is enabled prio to XDP program setup,
> this timestamp enablement will be overwritten when buffer size is
> written into SRRCTL register.
>
Hi all,
I'm actually searching for the root cause of a similar problem (RX
timestamp lost) that I can reproduce here, but the setup is slightly
different. The setup:
- igc driver
- i225/226 network card
- When starting to transmit frames using XDP with zero copy enabled
another application (ptp4l) complains about missing RX timestamps
- Loading the XDP program has already been removed (not needed for
TX only)
- ptp4l is using the traditional socket API
- The RX timestamps seem to come back once we stop sending frames
using XDP
The "zero copy support" enabled part is important. If ZC support is not
requested everything works fine.
Any ideas?
Best regards,
Florian
^ permalink raw reply [flat|nested] 9+ messages in thread
* [xdp-hints] Re: [PATCH net v3 1/1] igc: read before write to SRRCTL register
2023-04-15 9:19 ` Florian Bezdeka
@ 2023-04-16 2:19 ` Song, Yoong Siang
2023-04-17 2:53 ` Zulkifli, Muhammad Husaini
0 siblings, 1 reply; 9+ messages in thread
From: Song, Yoong Siang @ 2023-04-16 2:19 UTC (permalink / raw)
To: Bezdeka, Florian, Brandeburg, Jesse, Nguyen, Anthony L,
David S . Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Alexei Starovoitov, Daniel Borkmann, Jesper Dangaard Brouer,
John Fastabend, Fijalkowski, Maciej, Vedang Patel, Joseph, Jithu,
Andre Guedes, Brouer, Jesper, Stanislav Fomichev, Keller,
Jacob E, David Laight
Cc: intel-wired-lan, netdev, linux-kernel, bpf, xdp-hints, stable
On Saturday, April 15, 2023 5:20 PM, Florian Bezdeka <florian.bezdeka@siemens.com> wrote:
>On 14.04.23 17:49, Song Yoong Siang wrote:
>> igc_configure_rx_ring() function will be called as part of XDP program
>> setup. If Rx hardware timestamp is enabled prio to XDP program setup,
>> this timestamp enablement will be overwritten when buffer size is
>> written into SRRCTL register.
>>
>
>Hi all,
>
>I'm actually searching for the root cause of a similar problem (RX timestamp lost)
>that I can reproduce here, but the setup is slightly different. The setup:
>
>- igc driver
>- i225/226 network card
>- When starting to transmit frames using XDP with zero copy enabled
> another application (ptp4l) complains about missing RX timestamps
>- Loading the XDP program has already been removed (not needed for
> TX only)
>- ptp4l is using the traditional socket API
>- The RX timestamps seem to come back once we stop sending frames
> using XDP
>
>The "zero copy support" enabled part is important. If ZC support is not requested
>everything works fine.
>
>Any ideas?
>
>Best regards,
>Florian
>
Hi Florian,
You means this patch does not help on your issue?
Need to understand more on the setup and behavior to tell.
Are ptp4l and XDP ZC Tx apps running on same queue or separate queue?
I suggest you to run " sudo hwstamp_ctl -i eth0 -r 1 " multiple times in the middle
to check the behavior.
Thanks & Regards
Siang
^ permalink raw reply [flat|nested] 9+ messages in thread
* [xdp-hints] Re: [PATCH net v3 1/1] igc: read before write to SRRCTL register
2023-04-16 2:19 ` Song, Yoong Siang
@ 2023-04-17 2:53 ` Zulkifli, Muhammad Husaini
2023-04-17 18:50 ` Jakub Kicinski
0 siblings, 1 reply; 9+ messages in thread
From: Zulkifli, Muhammad Husaini @ 2023-04-17 2:53 UTC (permalink / raw)
To: Song, Yoong Siang, Bezdeka, Florian, Brandeburg, Jesse, Nguyen,
Anthony L, David S . Miller, Eric Dumazet, Jakub Kicinski,
Paolo Abeni, Alexei Starovoitov, Daniel Borkmann,
Jesper Dangaard Brouer, John Fastabend, Fijalkowski, Maciej,
Vedang Patel, Joseph, Jithu, Andre Guedes, Brouer, Jesper,
Stanislav Fomichev, Keller, Jacob E, David Laight
Cc: xdp-hints, netdev, linux-kernel, stable, intel-wired-lan, bpf
Hi Florian,
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of
> Song, Yoong Siang
> Sent: Sunday, 16 April, 2023 10:19 AM
> To: Bezdeka, Florian <florian.bezdeka@siemens.com>; Brandeburg, Jesse
> <jesse.brandeburg@intel.com>; Nguyen, Anthony L
> <anthony.l.nguyen@intel.com>; David S . Miller <davem@davemloft.net>;
> Eric Dumazet <edumazet@google.com>; Jakub Kicinski <kuba@kernel.org>;
> Paolo Abeni <pabeni@redhat.com>; Alexei Starovoitov <ast@kernel.org>;
> Daniel Borkmann <daniel@iogearbox.net>; Jesper Dangaard Brouer
> <hawk@kernel.org>; John Fastabend <john.fastabend@gmail.com>;
> Fijalkowski, Maciej <maciej.fijalkowski@intel.com>; Vedang Patel
> <vedang.patel@intel.com>; Joseph, Jithu <jithu.joseph@intel.com>; Andre
> Guedes <andre.guedes@intel.com>; Brouer, Jesper <brouer@redhat.com>;
> Stanislav Fomichev <sdf@google.com>; Keller, Jacob E
> <jacob.e.keller@intel.com>; David Laight <David.Laight@ACULAB.COM>
> Cc: xdp-hints@xdp-project.net; netdev@vger.kernel.org; linux-
> kernel@vger.kernel.org; stable@vger.kernel.org; intel-wired-
> lan@lists.osuosl.org; bpf@vger.kernel.org
> Subject: Re: [Intel-wired-lan] [PATCH net v3 1/1] igc: read before write to
> SRRCTL register
>
> On Saturday, April 15, 2023 5:20 PM, Florian Bezdeka
> <florian.bezdeka@siemens.com> wrote:
> >On 14.04.23 17:49, Song Yoong Siang wrote:
> >> igc_configure_rx_ring() function will be called as part of XDP
> >> program setup. If Rx hardware timestamp is enabled prio to XDP
> >> program setup, this timestamp enablement will be overwritten when
> >> buffer size is written into SRRCTL register.
> >>
> >
> >Hi all,
> >
> >I'm actually searching for the root cause of a similar problem (RX
> >timestamp lost) that I can reproduce here, but the setup is slightly
> different. The setup:
> >
> >- igc driver
> >- i225/226 network card
> >- When starting to transmit frames using XDP with zero copy enabled
> > another application (ptp4l) complains about missing RX timestamps
> >- Loading the XDP program has already been removed (not needed for
> > TX only)
> >- ptp4l is using the traditional socket API
> >- The RX timestamps seem to come back once we stop sending frames
> > using XDP
> >
> >The "zero copy support" enabled part is important. If ZC support is not
> >requested everything works fine.
> >
> >Any ideas?
Are you observing similar issue like below?
ptp4l: timed out while polling for tx timestamp
ptp4l: increasing tx_timestamp_timeout may correct this issue
If yes, only TXSTAMPO register is used for both PTP and non-PTP packets in
the current driver code. There is a possibility that the time stamp
for a PTP packet will be lost when there is a lot of traffic when multiple
applications request for hardware transmission timestamps.
Few months back, I submitted a patch series to enable the DMA
Timestamp for non-ptp packet which can resolve the above issue.
https://lore.kernel.org/netdev/20221018010733.4765-1-muhammad.husaini.zulkifli@intel.com/T/
Will continuing back the activity soon.
But you might want to try with series submitted by Vinicius as well.
This patch series add support for four sets of timestamping register.
https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20230228054534.1093483-2-vinicius.gomes@intel.com/
https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20230228054534.1093483-3-vinicius.gomes@intel.com/
https://patchwork.ozlabs.org/project/intel-wired-lan/patch/20230228054534.1093483-4-vinicius.gomes@intel.com/
Would it be possible for you to try any of the above patch series
and see if that fixes your problem?
Thanks,
Husaini
> >
> >Best regards,
> >Florian
> >
>
> Hi Florian,
>
> You means this patch does not help on your issue?
> Need to understand more on the setup and behavior to tell.
> Are ptp4l and XDP ZC Tx apps running on same queue or separate queue?
> I suggest you to run " sudo hwstamp_ctl -i eth0 -r 1 " multiple times in the
> middle to check the behavior.
>
> Thanks & Regards
> Siang
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan@osuosl.org
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 9+ messages in thread
* [xdp-hints] Re: [PATCH net v3 1/1] igc: read before write to SRRCTL register
2023-04-17 2:53 ` Zulkifli, Muhammad Husaini
@ 2023-04-17 18:50 ` Jakub Kicinski
0 siblings, 0 replies; 9+ messages in thread
From: Jakub Kicinski @ 2023-04-17 18:50 UTC (permalink / raw)
To: Zulkifli, Muhammad Husaini
Cc: Song, Yoong Siang, Bezdeka, Florian, Brandeburg, Jesse, Nguyen,
Anthony L, David S . Miller, Eric Dumazet, Paolo Abeni,
Alexei Starovoitov, Daniel Borkmann, Jesper Dangaard Brouer,
John Fastabend, Fijalkowski, Maciej, Vedang Patel, Joseph, Jithu,
Andre Guedes, Brouer, Jesper, Stanislav Fomichev, Keller,
Jacob E, David Laight, xdp-hints, netdev, linux-kernel, stable,
intel-wired-lan, bpf
On Mon, 17 Apr 2023 02:53:13 +0000 Zulkifli, Muhammad Husaini wrote:
> Are you observing similar issue like below?
> ptp4l: timed out while polling for tx timestamp
> ptp4l: increasing tx_timestamp_timeout may correct this issue
>
> If yes, only TXSTAMPO register is used for both PTP and non-PTP packets in
> the current driver code. There is a possibility that the time stamp
> for a PTP packet will be lost when there is a lot of traffic when multiple
> applications request for hardware transmission timestamps.
> Few months back, I submitted a patch series to enable the DMA
> Timestamp for non-ptp packet which can resolve the above issue.
> https://lore.kernel.org/netdev/20221018010733.4765-1-muhammad.husaini.zulkifli@intel.com/T/
> Will continuing back the activity soon.
FWIW the work on selecting the source of the timestamp is progressing
slowly:
https://lore.kernel.org/all/20230406173308.401924-1-kory.maincent@bootlin.com/
^ permalink raw reply [flat|nested] 9+ messages in thread
* [xdp-hints] Re: [Intel-wired-lan] [PATCH net v3 1/1] igc: read before write to SRRCTL register
2023-04-14 15:49 [xdp-hints] [PATCH net v3 1/1] igc: read before write to SRRCTL register Song Yoong Siang
2023-04-14 20:05 ` [xdp-hints] " Jesper Dangaard Brouer
2023-04-15 9:19 ` Florian Bezdeka
@ 2023-04-30 5:05 ` naamax.meir
2 siblings, 0 replies; 9+ messages in thread
From: naamax.meir @ 2023-04-30 5:05 UTC (permalink / raw)
To: Song Yoong Siang, Jesse Brandeburg, Tony Nguyen,
David S . Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Alexei Starovoitov, Daniel Borkmann, Jesper Dangaard Brouer,
John Fastabend, Maciej Fijalkowski, Vedang Patel, Jithu Joseph,
Andre Guedes, Jesper Dangaard Brouer, Stanislav Fomichev,
Jacob Keller, David Laight
Cc: xdp-hints, netdev, linux-kernel, stable, intel-wired-lan, bpf
On 4/14/2023 18:49, Song Yoong Siang wrote:
> igc_configure_rx_ring() function will be called as part of XDP program
> setup. If Rx hardware timestamp is enabled prio to XDP program setup,
> this timestamp enablement will be overwritten when buffer size is
> written into SRRCTL register.
>
> Thus, this commit read the register value before write to SRRCTL
> register. This commit is tested by using xdp_hw_metadata bpf selftest
> tool. The tool enables Rx hardware timestamp and then attach XDP program
> to igc driver. It will display hardware timestamp of UDP packet with
> port number 9092. Below are detail of test steps and results.
>
> Command on DUT:
> sudo ./xdp_hw_metadata <interface name>
>
> Command on Link Partner:
> echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
>
> Result before this patch:
> skb hwtstamp is not found!
>
> Result after this patch:
> found skb hwtstamp = 1677800973.642836757
>
> Optionally, read PHC to confirm the values obtained are almost the same:
> Command:
> sudo ./testptp -d /dev/ptp0 -g
> Result:
> clock time: 1677800973.913598978 or Fri Mar 3 07:49:33 2023
>
> Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy")
> Cc: <stable@vger.kernel.org> # 5.14+
> Signed-off-by: Song Yoong Siang <yoong.siang.song@intel.com>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Jesper Dangaard Brouer <brouer@redhat.com>
> ---
> v2 -> v3: Refactor SRRCTL definitions to more human readable definitions
> v1 -> v2: Fix indention
> ---
> drivers/net/ethernet/intel/igc/igc_base.h | 11 ++++++++---
> drivers/net/ethernet/intel/igc/igc_main.c | 7 +++++--
> 2 files changed, 13 insertions(+), 5 deletions(-)
Tested-by: Naama Meir <naamax.meir@linux.intel.com>
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-04-30 5:06 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-14 15:49 [xdp-hints] [PATCH net v3 1/1] igc: read before write to SRRCTL register Song Yoong Siang
2023-04-14 20:05 ` [xdp-hints] " Jesper Dangaard Brouer
2023-04-17 14:24 ` Jesper Dangaard Brouer
2023-04-18 9:31 ` Florian Bezdeka
2023-04-15 9:19 ` Florian Bezdeka
2023-04-16 2:19 ` Song, Yoong Siang
2023-04-17 2:53 ` Zulkifli, Muhammad Husaini
2023-04-17 18:50 ` Jakub Kicinski
2023-04-30 5:05 ` [xdp-hints] Re: [Intel-wired-lan] " naamax.meir
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox