XDP hardware hints discussion mail archive
 help / color / mirror / Atom feed
From: Stanislav Fomichev <sdf@google.com>
To: "Toke Høiland-Jørgensen" <toke@redhat.com>
Cc: Maciej Fijalkowski <maciej.fijalkowski@intel.com>,
	Jakub Kicinski <kuba@kernel.org>,
	bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net,
	andrii@kernel.org, martin.lau@linux.dev, song@kernel.org,
	yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org,
	haoluo@google.com, jolsa@kernel.org,
	Tariq Toukan <tariqt@nvidia.com>, David Ahern <dsahern@gmail.com>,
	Willem de Bruijn <willemb@google.com>,
	Jesper Dangaard Brouer <brouer@redhat.com>,
	Anatoly Burakov <anatoly.burakov@intel.com>,
	Alexander Lobakin <alexandr.lobakin@intel.com>,
	Magnus Karlsson <magnus.karlsson@gmail.com>,
	Maryam Tahhan <mtahhan@redhat.com>,
	xdp-hints@xdp-project.net, netdev@vger.kernel.org
Subject: [xdp-hints] Re: [PATCH bpf-next v2 6/8] mlx4: Introduce mlx4_xdp_buff wrapper for xdp_buff
Date: Mon, 28 Nov 2022 13:58:19 -0800	[thread overview]
Message-ID: <CAKH8qBvbYuCq-iiXnMw1QxFbfLFhorpF1+GGqU1yVzX2LhoUzQ@mail.gmail.com> (raw)
In-Reply-To: <874jun3m58.fsf@toke.dk>

On Thu, Nov 24, 2022 at 4:36 PM Toke Høiland-Jørgensen <toke@redhat.com> wrote:
>
> Maciej Fijalkowski <maciej.fijalkowski@intel.com> writes:
>
> > On Thu, Nov 24, 2022 at 04:17:01PM +0100, Maciej Fijalkowski wrote:
> >> On Thu, Nov 24, 2022 at 03:39:20PM +0100, Toke Høiland-Jørgensen wrote:
> >> > Jakub Kicinski <kuba@kernel.org> writes:
> >> >
> >> > > On Wed, 23 Nov 2022 22:55:21 +0100 Toke Høiland-Jørgensen wrote:
> >> > >> > Good idea, prototyped below, lmk if it that's not what you had in mind.
> >> > >> >
> >> > >> > struct xdp_buff_xsk {
> >> > >> >       struct xdp_buff            xdp;                  /*     0    56 */
> >> > >> >       u8                         cb[16];               /*    56    16 */
> >> > >> >       /* --- cacheline 1 boundary (64 bytes) was 8 bytes ago --- */
> >> > >>
> >> > >> As pahole helpfully says here, xdp_buff is actually only 8 bytes from
> >> > >> being a full cache line. I thought about adding a 'cb' field like this
> >> > >> to xdp_buff itself, but figured that since there's only room for a
> >> > >> single pointer, why not just add that and let the driver point it to
> >> > >> where it wants to store the extra context data?
> >> > >
> >> > > What if the driver wants to store multiple pointers or an integer or
> >> > > whatever else? The single pointer seems quite arbitrary and not
> >> > > strictly necessary.
> >> >
> >> > Well, then you allocate a separate struct and point to that? Like I did
> >> > in mlx5:
> >> >
> >> >
> >> > +  struct mlx5_xdp_ctx mlctx = { .cqe = cqe, .rq = rq };
> >> > +  struct xdp_buff xdp = { .drv_priv = &mlctx };
> >> >
> >> > but yeah, this does give an extra pointer deref on access. I'm not
> >> > really opposed to the cb field either, I just think it's a bit odd to
> >> > put it in struct xdp_buff_xsk; that basically requires the driver to
> >> > keep the layouts in sync.
> >> >
> >> > Instead, why not but a cb field into xdp_buff itself so it can be used
> >> > for both the XSK and the non-XSK paths? Then the driver can just
> >> > typecast the xdp_buff into its own struct that has whatever data it
> >> > wants in place of the cb field?

Agreed, maybe having an explicit cb field in the xdp_buff is a nice
compromise (assuming, over time, most devices will use it).

> >> Why can't you simply have a pointer to xdp_buff in driver specific
> >> xdp_buff container which would point to xdp_buff that is stack based (or
> >> whatever else memory that will back it up - I am about to push a change
> >> that makes ice driver embed xdp_buff within a struct that represents Rx
> >> ring) for XDP path and for ZC the pointer to xdp_buff that you get from
> >> xsk_buff_pool ? This would satisfy both sides I believe and would let us
> >> keep the same container struct.
> >>
> >> struct mlx4_xdp_buff {
> >>      struct xdp_buff *xdp;
> >>      struct mlx4_cqe *cqe;
> >>      struct mlx4_en_dev *mdev;
> >>      struct mlx4_en_rx_ring *ring;
> >>      struct net_device *dev;
> >> };
> >
> > Nah this won't work from kfunc POV, probably no way to retrieve the
> > mlx4_xdp_buff based on xdp_buff ptr that needs to be used as an arg.
> >
> > Sorry I'll think more about it, in the meantime let's hear more voices
> > whether we should keep Stan's original approach + modify xdp_buff_xsk or
> > go with Toke's proposal.
>
> OK, so I played around with the mlx5 code a bit more, and I think the
> "wrapping struct + cb area" can be made to work without too many ugly
> casts; I'll send an updated version of the mlx5 patches with this
> incorporated tomorrow, after I've run some tests...

I'll probably send a v3 sometime tomorrow (PST), so maybe wait for me
to make sure we are working on the same base?
Or LMK if you prefer to do it differently..

  reply	other threads:[~2022-11-28 21:58 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-21 18:25 [xdp-hints] [PATCH bpf-next v2 0/8] xdp: hints via kfuncs Stanislav Fomichev
2022-11-21 18:25 ` [xdp-hints] [PATCH bpf-next v2 1/8] bpf: Document XDP RX metadata Stanislav Fomichev
2022-11-21 18:25 ` [xdp-hints] [PATCH bpf-next v2 2/8] bpf: XDP metadata RX kfuncs Stanislav Fomichev
2022-11-21 23:31   ` [xdp-hints] " kernel test robot
2022-11-23  6:34   ` Martin KaFai Lau
2022-11-23 18:43     ` Stanislav Fomichev
2022-11-23 14:24   ` Toke Høiland-Jørgensen
2022-11-23 18:43     ` Stanislav Fomichev
2022-11-24  2:23   ` kernel test robot
2022-11-24 12:19   ` kernel test robot
2022-11-24 13:09   ` kernel test robot
2022-11-25 17:53   ` Toke Høiland-Jørgensen
2022-11-28 18:53     ` Stanislav Fomichev
2022-11-28 19:21       ` Stanislav Fomichev
2022-11-28 22:25         ` Toke Høiland-Jørgensen
2022-11-28 22:10       ` Toke Høiland-Jørgensen
2022-11-30 17:24   ` Larysa Zaremba
2022-11-30 19:06     ` Stanislav Fomichev
2022-11-30 20:17       ` Stanislav Fomichev
2022-12-01 13:52         ` Larysa Zaremba
2022-12-01 17:14           ` Stanislav Fomichev
2022-11-21 18:25 ` [xdp-hints] [PATCH bpf-next v2 3/8] veth: Introduce veth_xdp_buff wrapper for xdp_buff Stanislav Fomichev
2022-11-21 18:25 ` [xdp-hints] [PATCH bpf-next v2 4/8] veth: Support RX XDP metadata Stanislav Fomichev
2022-11-21 18:25 ` [xdp-hints] [PATCH bpf-next v2 5/8] selftests/bpf: Verify xdp_metadata xdp->af_xdp path Stanislav Fomichev
2022-11-29 10:06   ` [xdp-hints] " Anton Protopopov
2022-11-29 18:52     ` Stanislav Fomichev
2022-11-21 18:25 ` [xdp-hints] [PATCH bpf-next v2 6/8] mlx4: Introduce mlx4_xdp_buff wrapper for xdp_buff Stanislav Fomichev
2022-11-22 13:49   ` [xdp-hints] " Tariq Toukan
2022-11-22 18:08     ` Stanislav Fomichev
2022-11-23 14:33   ` Toke Høiland-Jørgensen
2022-11-23 18:26     ` Stanislav Fomichev
2022-11-23 19:14       ` Jakub Kicinski
2022-11-23 19:52         ` sdf
2022-11-23 21:54           ` Maciej Fijalkowski
2022-11-23 21:55           ` Toke Høiland-Jørgensen
2022-11-24  1:47             ` Jakub Kicinski
2022-11-24 14:39               ` Toke Høiland-Jørgensen
2022-11-24 15:17                 ` Maciej Fijalkowski
2022-11-24 16:11                   ` Maciej Fijalkowski
2022-11-25  0:36                     ` Toke Høiland-Jørgensen
2022-11-28 21:58                       ` Stanislav Fomichev [this message]
2022-11-28 22:11                         ` Toke Høiland-Jørgensen
2022-11-21 18:25 ` [xdp-hints] [PATCH bpf-next v2 7/8] mxl4: Support RX XDP metadata Stanislav Fomichev
2022-11-22 13:50   ` [xdp-hints] " Tariq Toukan
2022-11-22 18:08     ` Stanislav Fomichev
2022-11-21 18:25 ` [xdp-hints] [PATCH bpf-next v2 8/8] selftests/bpf: Simple program to dump XDP RX metadata Stanislav Fomichev
2022-11-23 14:26   ` [xdp-hints] " Toke Høiland-Jørgensen
2022-11-23 18:29     ` Stanislav Fomichev
2022-11-23 19:17       ` Jakub Kicinski
2022-11-23 19:54         ` Stanislav Fomichev
2022-11-23 14:46 ` [xdp-hints] [PATCH bpf-next 1/2] xdp: Add drv_priv pointer to struct xdp_buff Toke Høiland-Jørgensen
2022-11-23 14:46   ` [xdp-hints] [PATCH bpf-next 2/2] mlx5: Support XDP RX metadata Toke Høiland-Jørgensen
2022-11-23 22:29     ` [xdp-hints] " Saeed Mahameed
2022-11-23 22:44       ` Stanislav Fomichev

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.xdp-project.net/postorius/lists/xdp-hints.xdp-project.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAKH8qBvbYuCq-iiXnMw1QxFbfLFhorpF1+GGqU1yVzX2LhoUzQ@mail.gmail.com \
    --to=sdf@google.com \
    --cc=alexandr.lobakin@intel.com \
    --cc=anatoly.burakov@intel.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=daniel@iogearbox.net \
    --cc=dsahern@gmail.com \
    --cc=haoluo@google.com \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kpsingh@kernel.org \
    --cc=kuba@kernel.org \
    --cc=maciej.fijalkowski@intel.com \
    --cc=magnus.karlsson@gmail.com \
    --cc=martin.lau@linux.dev \
    --cc=mtahhan@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=song@kernel.org \
    --cc=tariqt@nvidia.com \
    --cc=toke@redhat.com \
    --cc=willemb@google.com \
    --cc=xdp-hints@xdp-project.net \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox