XDP hardware hints discussion mail archive
 help / color / mirror / Atom feed
From: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
To: Stanislav Fomichev <sdf@google.com>
Cc: Larysa Zaremba <larysa.zaremba@intel.com>,
	bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net,
	andrii@kernel.org, martin.lau@linux.dev, song@kernel.org,
	yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org,
	haoluo@google.com, jolsa@kernel.org,
	David Ahern <dsahern@gmail.com>, Jakub Kicinski <kuba@kernel.org>,
	Willem de Bruijn <willemb@google.com>,
	Jesper Dangaard Brouer <brouer@redhat.com>,
	Anatoly Burakov <anatoly.burakov@intel.com>,
	Alexander Lobakin <alexandr.lobakin@intel.com>,
	Magnus Karlsson <magnus.karlsson@gmail.com>,
	Maryam Tahhan <mtahhan@redhat.com>,
	xdp-hints@xdp-project.net, netdev@vger.kernel.org,
	Willem de Bruijn <willemdebruijn.kernel@gmail.com>,
	Alexei Starovoitov <alexei.starovoitov@gmail.com>,
	Simon Horman <simon.horman@corigine.com>,
	Tariq Toukan <tariqt@mellanox.com>,
	Saeed Mahameed <saeedm@mellanox.com>
Subject: [xdp-hints] Re: [RFC bpf-next 05/23] ice: Introduce ice_xdp_buff
Date: Thu, 7 Sep 2023 18:42:33 +0200	[thread overview]
Message-ID: <ZPn9eRofEv3guPLj@boxer> (raw)
In-Reply-To: <CAKH8qBuzgtJj=OKMdsxEkyML36VsAuZpcrsXcyqjdKXSJCBq=Q@mail.gmail.com>

On Thu, Sep 07, 2023 at 09:33:14AM -0700, Stanislav Fomichev wrote:
> On Thu, Sep 7, 2023 at 7:27 AM Larysa Zaremba <larysa.zaremba@intel.com> wrote:
> >
> > On Tue, Sep 05, 2023 at 07:53:03PM +0200, Maciej Fijalkowski wrote:
> > > On Mon, Sep 04, 2023 at 08:11:09PM +0200, Larysa Zaremba wrote:
> > > > On Mon, Sep 04, 2023 at 05:32:14PM +0200, Maciej Fijalkowski wrote:
> > > > > On Thu, Aug 24, 2023 at 09:26:44PM +0200, Larysa Zaremba wrote:
> > > > > > In order to use XDP hints via kfuncs we need to put
> > > > > > RX descriptor and ring pointers just next to xdp_buff.
> > > > > > Same as in hints implementations in other drivers, we achieve
> > > > > > this through putting xdp_buff into a child structure.
> > > > >
> > > > > Don't you mean a parent struct? xdp_buff will be 'child' of ice_xdp_buff
> > > > > if i'm reading this right.
> > > > >
> > > >
> > > > ice_xdp_buff is a child in terms of inheritance (pointer to ice_xdp_buff could
> > > > replace pointer to xdp_buff, but not in reverse).
> > > >
> > > > > >
> > > > > > Currently, xdp_buff is stored in the ring structure,
> > > > > > so replace it with union that includes child structure.
> > > > > > This way enough memory is available while existing XDP code
> > > > > > remains isolated from hints.
> > > > > >
> > > > > > Minimum size of the new child structure (ice_xdp_buff) is exactly
> > > > > > 64 bytes (single cache line). To place it at the start of a cache line,
> > > > > > move 'next' field from CL1 to CL3, as it isn't used often. This still
> > > > > > leaves 128 bits available in CL3 for packet context extensions.
> > > > >
> > > > > I believe ice_xdp_buff will be beefed up in later patches, so what is the
> > > > > point of moving 'next' ? We won't be able to keep ice_xdp_buff in a single
> > > > > CL anyway.
> > > > >
> > > >
> > > > It is to at least keep xdp_buff and descriptor pointer (used for every hint) in
> > > > a single CL, other fields are situational.
> > >
> > > Right, something must be moved...still, would be good to see perf
> > > before/after :)
> > >
> > > >
> > > > > >
> > > > > > Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
> > > > > > ---
> > > > > >  drivers/net/ethernet/intel/ice/ice_txrx.c     |  7 +++--
> > > > > >  drivers/net/ethernet/intel/ice/ice_txrx.h     | 26 ++++++++++++++++---
> > > > > >  drivers/net/ethernet/intel/ice/ice_txrx_lib.h | 10 +++++++
> > > > > >  3 files changed, 38 insertions(+), 5 deletions(-)
> > > > > >
> > > > > > diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
> > > > > > index 40f2f6dabb81..4e6546d9cf85 100644
> > > > > > --- a/drivers/net/ethernet/intel/ice/ice_txrx.c
> > > > > > +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
> > > > > > @@ -557,13 +557,14 @@ ice_rx_frame_truesize(struct ice_rx_ring *rx_ring, const unsigned int size)
> > > > > >   * @xdp_prog: XDP program to run
> > > > > >   * @xdp_ring: ring to be used for XDP_TX action
> > > > > >   * @rx_buf: Rx buffer to store the XDP action
> > > > > > + * @eop_desc: Last descriptor in packet to read metadata from
> > > > > >   *
> > > > > >   * Returns any of ICE_XDP_{PASS, CONSUMED, TX, REDIR}
> > > > > >   */
> > > > > >  static void
> > > > > >  ice_run_xdp(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp,
> > > > > >             struct bpf_prog *xdp_prog, struct ice_tx_ring *xdp_ring,
> > > > > > -           struct ice_rx_buf *rx_buf)
> > > > > > +           struct ice_rx_buf *rx_buf, union ice_32b_rx_flex_desc *eop_desc)
> > > > > >  {
> > > > > >         unsigned int ret = ICE_XDP_PASS;
> > > > > >         u32 act;
> > > > > > @@ -571,6 +572,8 @@ ice_run_xdp(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp,
> > > > > >         if (!xdp_prog)
> > > > > >                 goto exit;
> > > > > >
> > > > > > +       ice_xdp_meta_set_desc(xdp, eop_desc);
> > > > >
> > > > > I am currently not sure if for multi-buffer case HW repeats all the
> > > > > necessary info within each descriptor for every frag? IOW shouldn't you be
> > > > > using the ice_rx_ring::first_desc?
> > > > >
> > > > > Would be good to test hints for mbuf case for sure.
> > > > >
> > > >
> > > > In the skb path, we take metadata from the last descriptor only, so this should
> > > > be fine. Really worth testing with mbuf though.
> >
> > I retract my promise to test this with mbuf, as for now hints and mbuf are not
> > supposed to go together [0].
> 
> Hm, I don't think it's intentional. I don't see why mbuf and hints
> can't coexist.

They should coexist, xdp mbuf support is an integral part of driver as we
know:)

> Anything pops into your mind? Otherwise, can change that mask to be
> ~(BPF_F_XDP_DEV_BOUND_ONLY|BPF_F_XDP_HAS_FRAGS) as part of the series
> (or separately, up to you).

+1

> 
> > Making sure they can co-exist peacefully can be a topic for another series.
> > For now I just can just say with high confidence that in case of multi-buffer
> > frames, we do have all the supported metadata in the EoP descriptor.
> >
> > [0] https://elixir.bootlin.com/linux/v6.5.2/source/kernel/bpf/offload.c#L234
> >
> > >
> > > Ok, thanks!
> > >
> 

  reply	other threads:[~2023-09-07 16:42 UTC|newest]

Thread overview: 70+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-08-24 19:26 [xdp-hints] [RFC bpf-next 00/23] XDP metadata via kfuncs for ice + mlx5 Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 01/23] ice: make RX hash reading code more reusable Larysa Zaremba
2023-09-04 14:37   ` [xdp-hints] " Maciej Fijalkowski
2023-09-06 12:23     ` Alexander Lobakin
2023-09-14 16:12   ` Alexander Lobakin
2023-09-14 16:15     ` Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 02/23] ice: make RX HW timestamp " Larysa Zaremba
2023-09-04 14:56   ` [xdp-hints] " Maciej Fijalkowski
2023-09-04 16:29     ` Larysa Zaremba
2023-09-05 15:22       ` Maciej Fijalkowski
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 03/23] ice: make RX checksum checking " Larysa Zaremba
2023-09-04 15:02   ` [xdp-hints] " Maciej Fijalkowski
2023-09-04 18:01     ` Larysa Zaremba
2023-09-05 15:37       ` Maciej Fijalkowski
2023-09-05 16:53         ` Larysa Zaremba
2023-09-05 17:44           ` Maciej Fijalkowski
2023-09-06  9:28             ` Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 04/23] ice: Make ptype internal to descriptor info processing Larysa Zaremba
2023-09-04 15:04   ` [xdp-hints] " Maciej Fijalkowski
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 05/23] ice: Introduce ice_xdp_buff Larysa Zaremba
2023-09-04 15:32   ` [xdp-hints] " Maciej Fijalkowski
2023-09-04 18:11     ` Larysa Zaremba
2023-09-05 17:53       ` Maciej Fijalkowski
2023-09-07 14:21         ` Larysa Zaremba
2023-09-07 16:33           ` Stanislav Fomichev
2023-09-07 16:42             ` Maciej Fijalkowski [this message]
2023-09-07 16:43               ` Maciej Fijalkowski
2023-09-13 15:40                 ` Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 06/23] ice: Support HW timestamp hint Larysa Zaremba
2023-09-04 15:38   ` [xdp-hints] " Maciej Fijalkowski
2023-09-04 18:12     ` Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 07/23] ice: Support RX hash XDP hint Larysa Zaremba
2023-09-05 15:42   ` [xdp-hints] " Maciej Fijalkowski
2023-09-05 17:09     ` Larysa Zaremba
2023-09-06 12:03     ` Alexander Lobakin
2023-09-14 16:54   ` Alexander Lobakin
2023-09-14 16:59     ` Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 08/23] ice: Support XDP hints in AF_XDP ZC mode Larysa Zaremba
2023-09-04 15:42   ` [xdp-hints] " Maciej Fijalkowski
2023-09-04 18:14     ` Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 09/23] xdp: Add VLAN tag hint Larysa Zaremba
2023-09-14 16:18   ` [xdp-hints] " Alexander Lobakin
2023-09-14 16:21     ` Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 10/23] ice: Implement " Larysa Zaremba
2023-09-04 16:00   ` [xdp-hints] " Maciej Fijalkowski
2023-09-04 18:18     ` Larysa Zaremba
2023-09-14 16:25   ` Alexander Lobakin
2023-09-14 16:28     ` Larysa Zaremba
2023-09-14 16:38       ` Alexander Lobakin
2023-09-14 17:02         ` Larysa Zaremba
2023-09-18 14:07         ` Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 11/23] ice: use VLAN proto from ring packet context in skb path Larysa Zaremba
2023-09-14 16:30   ` [xdp-hints] " Alexander Lobakin
2023-09-14 16:30     ` Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 12/23] xdp: Add checksum hint Larysa Zaremba
2023-09-14 16:34   ` [xdp-hints] " Alexander Lobakin
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 13/23] ice: Implement " Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 14/23] selftests/bpf: Allow VLAN packets in xdp_hw_metadata Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 15/23] net, xdp: allow metadata > 32 Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 16/23] selftests/bpf: Add flags and new hints to xdp_hw_metadata Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 17/23] veth: Implement VLAN tag and checksum XDP hint Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 18/23] net: make vlan_get_tag() return -ENODATA instead of -EINVAL Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 19/23] selftests/bpf: Use AF_INET for TX in xdp_metadata Larysa Zaremba
2023-08-24 19:26 ` [xdp-hints] [RFC bpf-next 20/23] selftests/bpf: Check VLAN tag and proto " Larysa Zaremba
2023-08-24 19:27 ` [xdp-hints] [RFC bpf-next 21/23] selftests/bpf: check checksum state " Larysa Zaremba
2023-08-24 19:27 ` [xdp-hints] [RFC bpf-next 22/23] mlx5: implement VLAN tag XDP hint Larysa Zaremba
2023-08-24 19:27 ` [xdp-hints] [RFC bpf-next 23/23] mlx5: implement RX checksum " Larysa Zaremba
2023-08-31 14:50 ` [xdp-hints] Re: [RFC bpf-next 00/23] XDP metadata via kfuncs for ice + mlx5 Larysa Zaremba
2023-09-04 16:06 ` Maciej Fijalkowski
2023-09-06 14:09   ` Larysa Zaremba

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.xdp-project.net/postorius/lists/xdp-hints.xdp-project.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ZPn9eRofEv3guPLj@boxer \
    --to=maciej.fijalkowski@intel.com \
    --cc=alexandr.lobakin@intel.com \
    --cc=alexei.starovoitov@gmail.com \
    --cc=anatoly.burakov@intel.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=daniel@iogearbox.net \
    --cc=dsahern@gmail.com \
    --cc=haoluo@google.com \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kpsingh@kernel.org \
    --cc=kuba@kernel.org \
    --cc=larysa.zaremba@intel.com \
    --cc=magnus.karlsson@gmail.com \
    --cc=martin.lau@linux.dev \
    --cc=mtahhan@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=saeedm@mellanox.com \
    --cc=sdf@google.com \
    --cc=simon.horman@corigine.com \
    --cc=song@kernel.org \
    --cc=tariqt@mellanox.com \
    --cc=willemb@google.com \
    --cc=willemdebruijn.kernel@gmail.com \
    --cc=xdp-hints@xdp-project.net \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox