XDP hardware hints discussion mail archive
 help / color / mirror / Atom feed
From: sdf@google.com
To: Maryam Tahhan <mtahhan@redhat.com>
Cc: "Toke Høiland-Jørgensen" <toke@redhat.com>,
	"Jakub Kicinski" <kuba@kernel.org>,
	"Martin KaFai Lau" <martin.lau@linux.dev>,
	"Jesper Dangaard Brouer" <jbrouer@redhat.com>,
	brouer@redhat.com, bpf@vger.kernel.org, netdev@vger.kernel.org,
	xdp-hints@xdp-project.net, larysa.zaremba@intel.com,
	memxor@gmail.com, "Lorenzo Bianconi" <lorenzo@kernel.org>,
	"Alexei Starovoitov" <alexei.starovoitov@gmail.com>,
	"Daniel Borkmann" <borkmann@iogearbox.net>,
	"Andrii Nakryiko" <andrii.nakryiko@gmail.com>,
	dave@dtucker.co.uk, "Magnus Karlsson" <magnus.karlsson@intel.com>,
	bjorn@kernel.org
Subject: [xdp-hints] Re: [PATCH RFCv2 bpf-next 00/18] XDP-hints: XDP gaining access to HW offload hints via BTF
Date: Thu, 6 Oct 2022 10:22:26 -0700	[thread overview]
Message-ID: <Yz8O0rn7ps/m8iGi@google.com> (raw)
In-Reply-To: <be3f16a2-8422-4a34-3eb9-3943753d453e@redhat.com>

On 10/06, Maryam Tahhan wrote:
> On 05/10/2022 19:47, sdf@google.com wrote:
> > On 10/05, Toke H�iland-J�rgensen wrote:
> > > Stanislav Fomichev <sdf@google.com> writes:
> >
> > > > On Tue, Oct 4, 2022 at 5:59 PM Jakub Kicinski <kuba@kernel.org>  
> wrote:
> > > >>
> > > >> On Tue, 4 Oct 2022 17:25:51 -0700 Martin KaFai Lau wrote:
> > > >> > A intentionally wild question, what does it take for the driver
> > > to return the
> > > >> > hints.  Is the rx_desc and rx_queue enough?  When the xdp prog
> > > is calling a
> > > >> > kfunc/bpf-helper, like 'hwtstamp = bpf_xdp_get_hwtstamp()', can
> > > the driver
> > > >> > replace it with some inline bpf code (like how the inline code
> > > is generated for
> > > >> > the map_lookup helper).  The xdp prog can then store the
> > > hwstamp in the meta
> > > >> > area in any layout it wants.
> > > >>
> > > >> Since you mentioned it... FWIW that was always my preference
> > > rather than
> > > >> the BTF magic :)  The jited image would have to be per-driver like  
> we
> > > >> do for BPF offload but that's easy to do from the technical
> > > >> perspective (I doubt many deployments bind the same prog to  
> multiple
> > > >> HW devices)..
> > > >
> > > > +1, sounds like a good alternative (got your reply while typing)
> > > > I'm not too versed in the rx_desc/rx_queue area, but seems like  
> worst
> > > > case that bpf_xdp_get_hwtstamp can probably receive a xdp_md ctx and
> > > > parse it out from the pre-populated metadata?
> > > >
> > > > Btw, do we also need to think about the redirect case? What happens
> > > > when I redirect one frame from a device A with one metadata format  
> to
> > > > a device B with another?
> >
> > > Yes, we absolutely do! In fact, to me this (redirects) is the main
> > > reason why we need the ID in the packet in the first place: when  
> running
> > > on (say) a veth, an XDP program needs to be able to deal with packets
> > > from multiple physical NICs.
> >
> > > As far as API is concerned, my hope was that we could solve this with  
> a
> > > CO-RE like approach where the program author just writes something  
> like:
> >
> > > hw_tstamp = bpf_get_xdp_hint("hw_tstamp", u64);
> >
> > > and bpf_get_xdp_hint() is really a macro (or a special kind of
> > > relocation?) and libbpf would do the following on load:
> >
> > > - query the kernel BTF for all possible xdp_hint structs
> > > - figure out which of them have an 'u64 hw_tstamp' member
> > > - generate the necessary conditionals / jump table to disambiguate on
> > >    the BTF_ID in the packet
> >
> >
> > > Now, if this is better done by a kfunc I'm not terribly opposed to  
> that
> > > either, but I'm not sure it's actually better/easier to do in the  
> kernel
> > > than in libbpf at load time?
> >
> > Replied in the other thread, but to reiterate here: then btf_id in the
> > metadata has to stay and we either pre-generate those bpf_get_xdp_hint()
> > at libbpf or at kfunc load time level as you mention.
> >
> > But the program essentially has to handle all possible hints' btf ids
> > thrown
> > at it by the system. Not sure about the performance in this case :-/
> > Maybe that's something that can be hidden behind "I might receive  
> forwarded
> > packets and I know how to handle all metadata format" flag? By default,
> > we'll pre-generate parsing only for that specific device?

> I did a simple POC of Jespers xdp-hints with AF-XDP and CNDP (Cloud Native
> Data Plane). In the cases where my app had access to the HW I didn't need  
> to
> handle all possible hints... I knew what Drivers were on the system and  
> they
> were the hints I needed to deal with.

> So at program init time I registered the relevant BTF_IDs (and some  
> callback
> functions to handle them) from the NICs that were available to me in a
> simple tailq (tbh there were so few I could've probably used a static
> array).

> When processing the hints then I only needed to invoke the appropriate
> callback function based on the received BTF_ID. I didn't have a massive
> chains of if...else if... else statements.

> In the case where we have redirection to a virtual NIC and we don't
> necessarily know the underlying hints that are exposed to the app, could  
> we
> not still use the xdp_hints (as proposed by Jesper) themselves to indicate
> the relevant drivers to the application? or even indicate them via a map  
> or
> something?


Ideally this all should be handled by the common infra (libbpf/libxdp?).
We probably don't want every xdp/af_xdp user to custom-implement all this
btf_id->layout parsing? That's why the request for a selftest that shows
how metadata can be accessed from bpf/af_xdp.

  reply	other threads:[~2022-10-06 17:22 UTC|newest]

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-09-07 15:45 [xdp-hints] " Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 01/18] libbpf: factor out BTF loading from load_module_btfs() Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 02/18] libbpf: try to load vmlinux BTF from the kernel first Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 03/18] libbpf: patch module BTF obj+type ID into BPF insns Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 04/18] net: create xdp_hints_common and set functions Jesper Dangaard Brouer
2022-09-09 10:49   ` [xdp-hints] " Burakov, Anatoly
2022-09-09 14:13     ` Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 05/18] net: add net_device feature flag for XDP-hints Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 06/18] xdp: controlling XDP-hints from BPF-prog via helper Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 07/18] i40e: Refactor i40e_ptp_rx_hwtstamp Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 08/18] i40e: refactor i40e_rx_checksum with helper Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 09/18] bpf: export btf functions for modules Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 10/18] btf: Add helper for kernel modules to lookup full BTF ID Jesper Dangaard Brouer
2022-09-07 15:45 ` [xdp-hints] [PATCH RFCv2 bpf-next 11/18] i40e: add XDP-hints handling Jesper Dangaard Brouer
2022-09-07 15:46 ` [xdp-hints] [PATCH RFCv2 bpf-next 12/18] net: use XDP-hints in xdp_frame to SKB conversion Jesper Dangaard Brouer
2022-09-07 15:46 ` [xdp-hints] [PATCH RFCv2 bpf-next 13/18] mvneta: add XDP-hints support Jesper Dangaard Brouer
2022-09-07 15:46 ` [xdp-hints] [PATCH RFCv2 bpf-next 14/18] i40e: Add xdp_hints_union Jesper Dangaard Brouer
2022-09-07 15:46 ` [xdp-hints] [PATCH RFCv2 bpf-next 15/18] ixgbe: enable xdp-hints Jesper Dangaard Brouer
2022-09-07 15:46 ` [xdp-hints] [PATCH RFCv2 bpf-next 16/18] ixgbe: add rx timestamp xdp hints support Jesper Dangaard Brouer
2022-09-07 15:46 ` [xdp-hints] [PATCH RFCv2 bpf-next 17/18] xsk: AF_XDP xdp-hints support in desc options Jesper Dangaard Brouer
2022-09-08  8:06   ` [xdp-hints] " Magnus Karlsson
2022-09-08 10:10     ` Maryam Tahhan
2022-09-08 15:04       ` Jesper Dangaard Brouer
2022-09-09  6:43         ` Magnus Karlsson
2022-09-09  8:12           ` Maryam Tahhan
2022-09-09  9:42             ` Jesper Dangaard Brouer
2022-09-09 10:14               ` Magnus Karlsson
2022-09-09 12:35                 ` Jesper Dangaard Brouer
2022-09-09 12:44                   ` Magnus Karlsson
2022-09-07 15:46 ` [xdp-hints] [PATCH RFCv2 bpf-next 18/18] ixgbe: AF_XDP xdp-hints processing in ixgbe_clean_rx_irq_zc Jesper Dangaard Brouer
2022-09-08  9:30 ` [xdp-hints] Re: [PATCH RFCv2 bpf-next 00/18] XDP-hints: XDP gaining access to HW offload hints via BTF Alexander Lobakin
2022-09-09 13:48   ` Jesper Dangaard Brouer
2022-10-03 23:55 ` sdf
2022-10-04  9:29   ` Jesper Dangaard Brouer
2022-10-04 18:26     ` Stanislav Fomichev
2022-10-05  0:25       ` Martin KaFai Lau
2022-10-05  0:59         ` Jakub Kicinski
2022-10-05  1:02           ` Stanislav Fomichev
2022-10-05  1:24             ` Jakub Kicinski
2022-10-05  2:15               ` Stanislav Fomichev
2022-10-05 19:26                 ` Martin KaFai Lau
2022-10-06  9:14                   ` Magnus Karlsson
2022-10-06 15:29                     ` Jesper Dangaard Brouer
2022-10-11  6:29                       ` Martin KaFai Lau
2022-10-11 11:57                         ` Jesper Dangaard Brouer
2022-10-05 10:06             ` Toke Høiland-Jørgensen
2022-10-05 18:47               ` sdf
2022-10-06  8:19                 ` Maryam Tahhan
2022-10-06 17:22                   ` sdf [this message]
2022-10-05 14:19             ` Jesper Dangaard Brouer
2022-10-06 14:59               ` Jakub Kicinski
2022-10-05 13:43         ` Jesper Dangaard Brouer
2022-10-05 16:29       ` Jesper Dangaard Brouer
2022-10-05 18:43         ` sdf
2022-10-06 17:47           ` Jesper Dangaard Brouer
2022-10-07 15:05             ` David Ahern
2022-10-05 13:14     ` Burakov, Anatoly

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.xdp-project.net/postorius/lists/xdp-hints.xdp-project.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Yz8O0rn7ps/m8iGi@google.com \
    --to=sdf@google.com \
    --cc=alexei.starovoitov@gmail.com \
    --cc=andrii.nakryiko@gmail.com \
    --cc=bjorn@kernel.org \
    --cc=borkmann@iogearbox.net \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=dave@dtucker.co.uk \
    --cc=jbrouer@redhat.com \
    --cc=kuba@kernel.org \
    --cc=larysa.zaremba@intel.com \
    --cc=lorenzo@kernel.org \
    --cc=magnus.karlsson@intel.com \
    --cc=martin.lau@linux.dev \
    --cc=memxor@gmail.com \
    --cc=mtahhan@redhat.com \
    --cc=netdev@vger.kernel.org \
    --cc=toke@redhat.com \
    --cc=xdp-hints@xdp-project.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox