XDP hardware hints discussion mail archive
 help / color / mirror / Atom feed
From: "Toke Høiland-Jørgensen" <toke@redhat.com>
To: Stanislav Fomichev <sdf@google.com>,
	Martin KaFai Lau <martin.lau@linux.dev>
Cc: "Bezdeka, Florian" <florian.bezdeka@siemens.com>,
	"kuba@kernel.org" <kuba@kernel.org>,
	"john.fastabend@gmail.com" <john.fastabend@gmail.com>,
	"alexandr.lobakin@intel.com" <alexandr.lobakin@intel.com>,
	"anatoly.burakov@intel.com" <anatoly.burakov@intel.com>,
	"song@kernel.org" <song@kernel.org>,
	"Deric, Nemanja" <nemanja.deric@siemens.com>,
	"andrii@kernel.org" <andrii@kernel.org>,
	"Kiszka, Jan" <jan.kiszka@siemens.com>,
	"magnus.karlsson@gmail.com" <magnus.karlsson@gmail.com>,
	"willemb@google.com" <willemb@google.com>,
	"ast@kernel.org" <ast@kernel.org>,
	"brouer@redhat.com" <brouer@redhat.com>,
	"yhs@fb.com" <yhs@fb.com>,
	"kpsingh@kernel.org" <kpsingh@kernel.org>,
	"daniel@iogearbox.net" <daniel@iogearbox.net>,
	"bpf@vger.kernel.org" <bpf@vger.kernel.org>,
	"mtahhan@redhat.com" <mtahhan@redhat.com>,
	"xdp-hints@xdp-project.net" <xdp-hints@xdp-project.net>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"jolsa@kernel.org" <jolsa@kernel.org>,
	"haoluo@google.com" <haoluo@google.com>
Subject: [xdp-hints] Re: [RFC bpf-next 0/5] xdp: hints via kfuncs
Date: Tue, 01 Nov 2022 13:52:55 +0100	[thread overview]
Message-ID: <87wn8e4z14.fsf@toke.dk> (raw)
In-Reply-To: <CAKH8qBsGB1G60cu91Au816gsB2zF8T0P-yDwxbTEOxX0TN3WgA@mail.gmail.com>

Stanislav Fomichev <sdf@google.com> writes:

> On Mon, Oct 31, 2022 at 3:57 PM Martin KaFai Lau <martin.lau@linux.dev> wrote:
>>
>> On 10/31/22 10:00 AM, Stanislav Fomichev wrote:
>> >> 2. AF_XDP programs won't be able to access the metadata without using a
>> >> custom XDP program that calls the kfuncs and puts the data into the
>> >> metadata area. We could solve this with some code in libxdp, though; if
>> >> this code can be made generic enough (so it just dumps the available
>> >> metadata functions from the running kernel at load time), it may be
>> >> possible to make it generic enough that it will be forward-compatible
>> >> with new versions of the kernel that add new fields, which should
>> >> alleviate Florian's concern about keeping things in sync.
>> >
>> > Good point. I had to convert to a custom program to use the kfuncs :-(
>> > But your suggestion sounds good; maybe libxdp can accept some extra
>> > info about at which offset the user would like to place the metadata
>> > and the library can generate the required bytecode?
>> >
>> >> 3. It will make it harder to consume the metadata when building SKBs. I
>> >> think the CPUMAP and veth use cases are also quite important, and that
>> >> we want metadata to be available for building SKBs in this path. Maybe
>> >> this can be resolved by having a convenient kfunc for this that can be
>> >> used for programs doing such redirects. E.g., you could just call
>> >> xdp_copy_metadata_for_skb() before doing the bpf_redirect, and that
>> >> would recursively expand into all the kfunc calls needed to extract the
>> >> metadata supported by the SKB path?
>> >
>> > So this xdp_copy_metadata_for_skb will create a metadata layout that
>>
>> Can the xdp_copy_metadata_for_skb be written as a bpf prog itself?
>> Not sure where is the best point to specify this prog though.  Somehow during
>> bpf_xdp_redirect_map?
>> or this prog belongs to the target cpumap and the xdp prog redirecting to this
>> cpumap has to write the meta layout in a way that the cpumap is expecting?
>
> We're probably interested in triggering it from the places where xdp
> frames can eventually be converted into skbs?
> So for plain 'return XDP_PASS' and things like bpf_redirect/etc? (IOW,
> anything that's not XDP_DROP / AF_XDP redirect).
> We can probably make it magically work, and can generate
> kernel-digestible metadata whenever data == data_meta, but the
> question - should we?
> (need to make sure we won't regress any existing cases that are not
> relying on the metadata)

So I was thinking about whether we could have the kernel do this
automatically, and concluded that this was probably not feasible in
general, which is why I suggested the explicit helper. My reasoning was
as follows:

For straight XDP_PASS in the driver we don't actually need to do
anything today, as the driver itself will build the SKB and read any
metadata it needs from the HW descriptor[0].

This leaves packets that are redirected (either to a veth or a cpumap so
we build SKBs from them later); here the problem is that we buffer the
packets (for performance reasons) so that the redirect doesn't actually
happen until after the driver exits the NAPI loop. At which point we
don't have access to the HW descriptors anymore, so we can't actually
read the metadata.

This means that if we want to execute the metadata gathering
automatically, we'd have to do it in xdp_do_redirect(). Which means that
we'll have to figure out, at that point, whether the XDP frame is likely
to be converted to an SKB. This will add at least one branch (and
probably more) that will be in-path for every redirected frame.

Hence, making it up to the XDP program itself to decide whether it will
need the metadata for SKB conversion seems like a better choice, as long
as we make it easy for the XDP program to do this. Instead of a helper,
this could also simply be a new flag to the bpf_redirect{,_map}()
helpers (either opt-in or opt-out depending on the overhead), which
would be even simpler?

I.e.,

return bpf_redirect_map(&cpumap, 0, BPF_F_PREPARE_SKB_METADATA);

-Toke


[0] As an aside, in the future drivers may want to take advantage of the
XDP-specific metadata reading also when building SKBs (so it doesn't
have to implement it in both BPF and C code). For this, we could expose
a new internal helper function that the drivers could call to simply
execute the XDP-to-skb metadata helpers the same way the stack/helper
does.


  reply	other threads:[~2022-11-01 12:53 UTC|newest]

Thread overview: 50+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-27 20:00 [xdp-hints] " Stanislav Fomichev
2022-10-27 20:00 ` [xdp-hints] [RFC bpf-next 1/5] bpf: Support inlined/unrolled kfuncs for xdp metadata Stanislav Fomichev
2022-10-27 20:00 ` [xdp-hints] [RFC bpf-next 2/5] veth: Support rx timestamp metadata for xdp Stanislav Fomichev
2022-10-28  8:40   ` [xdp-hints] " Jesper Dangaard Brouer
2022-10-28 18:46     ` Stanislav Fomichev
2022-10-27 20:00 ` [xdp-hints] [RFC bpf-next 3/5] libbpf: Pass prog_ifindex via bpf_object_open_opts Stanislav Fomichev
2022-10-27 20:05   ` [xdp-hints] " Andrii Nakryiko
2022-10-27 20:10     ` Stanislav Fomichev
2022-10-27 20:00 ` [xdp-hints] [RFC bpf-next 4/5] selftests/bpf: Convert xskxceiver to use custom program Stanislav Fomichev
2022-10-27 20:00 ` [xdp-hints] [RFC bpf-next 5/5] selftests/bpf: Test rx_timestamp metadata in xskxceiver Stanislav Fomichev
2022-10-28  6:22   ` [xdp-hints] " Martin KaFai Lau
2022-10-28 10:37     ` Jesper Dangaard Brouer
2022-10-28 18:46       ` Stanislav Fomichev
2022-10-31 14:20         ` Alexander Lobakin
2022-10-31 14:29           ` Alexander Lobakin
2022-10-31 17:00           ` Stanislav Fomichev
2022-11-01 13:18             ` Jesper Dangaard Brouer
2022-11-01 20:12               ` Stanislav Fomichev
2022-11-01 22:23               ` Toke Høiland-Jørgensen
2022-10-28 15:58 ` [xdp-hints] Re: [RFC bpf-next 0/5] xdp: hints via kfuncs John Fastabend
2022-10-28 18:04   ` Jakub Kicinski
2022-10-28 18:46     ` Stanislav Fomichev
2022-10-28 23:16       ` John Fastabend
2022-10-29  1:14         ` Jakub Kicinski
2022-10-31 14:10           ` Bezdeka, Florian
2022-10-31 15:28             ` Toke Høiland-Jørgensen
2022-10-31 17:00               ` Stanislav Fomichev
2022-10-31 22:57                 ` Martin KaFai Lau
2022-11-01  1:59                   ` Stanislav Fomichev
2022-11-01 12:52                     ` Toke Høiland-Jørgensen [this message]
2022-11-01 13:43                       ` David Ahern
2022-11-01 14:20                         ` Toke Høiland-Jørgensen
2022-11-01 17:05                     ` Martin KaFai Lau
2022-11-01 20:12                       ` Stanislav Fomichev
2022-11-02 14:06                       ` Jesper Dangaard Brouer
2022-11-02 22:01                         ` Toke Høiland-Jørgensen
2022-11-02 23:10                           ` Stanislav Fomichev
2022-11-03  0:09                             ` Toke Høiland-Jørgensen
2022-11-03 12:01                               ` Jesper Dangaard Brouer
2022-11-03 12:48                                 ` Toke Høiland-Jørgensen
2022-11-03 15:25                                   ` Jesper Dangaard Brouer
2022-10-31 19:36               ` Yonghong Song
2022-10-31 22:09                 ` Stanislav Fomichev
2022-10-31 22:38                   ` Yonghong Song
2022-10-31 22:55                     ` Stanislav Fomichev
2022-11-01 14:23                       ` Jesper Dangaard Brouer
2022-11-01 17:31                   ` Martin KaFai Lau
2022-11-01 20:12                     ` Stanislav Fomichev
2022-11-01 21:17                       ` Martin KaFai Lau
2022-10-31 17:01           ` John Fastabend

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.xdp-project.net/postorius/lists/xdp-hints.xdp-project.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87wn8e4z14.fsf@toke.dk \
    --to=toke@redhat.com \
    --cc=alexandr.lobakin@intel.com \
    --cc=anatoly.burakov@intel.com \
    --cc=andrii@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=daniel@iogearbox.net \
    --cc=florian.bezdeka@siemens.com \
    --cc=haoluo@google.com \
    --cc=jan.kiszka@siemens.com \
    --cc=john.fastabend@gmail.com \
    --cc=jolsa@kernel.org \
    --cc=kpsingh@kernel.org \
    --cc=kuba@kernel.org \
    --cc=magnus.karlsson@gmail.com \
    --cc=martin.lau@linux.dev \
    --cc=mtahhan@redhat.com \
    --cc=nemanja.deric@siemens.com \
    --cc=netdev@vger.kernel.org \
    --cc=sdf@google.com \
    --cc=song@kernel.org \
    --cc=willemb@google.com \
    --cc=xdp-hints@xdp-project.net \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox