Re: XDP-hints: Howto support multiple BTF types per packet basis?

[Alexander Lobakin <alexandr.lobakin@intel.com>]

From: Toke Høiland-Jørgensen <toke@redhat.com>
Date: Fri, 28 May 2021 11:16:44 +0200

> John Fastabend <john.fastabend@gmail.com> writes:
> 
> >> > > union and independent set of BTFs are two different things, I'll let
> >> > > you guys figure out which one you need, but I replied how it could
> >> > > look like in CO-RE world
> >> >
> >> > I think a union is sufficient and more aligned with how the
> >> > hardware would actually work.
> >> 
> >> Sure. And I think those are two orthogonal concerns. You can start
> >> with a single struct mynic_metadata with union inside it, and later
> >> add the ability to swap mynic_metadata with another
> >> mynic_metadata___v2 that will have a similar union but with a
> >> different layout.
> >
> > Right and then you just have normal upgrade/downgrade problems with
> > any struct.
> >
> > Seems like a workable path to me. But, need to circle back to the
> > what we want to do with it part that Jesper replied to.
> 
> So while this seems to be a viable path for getting libbpf to do all the
> relocations (and thanks for hashing that out, I did not have a good grip
> of the details), doing it all in userspace means that there is no way
> for the XDP program to react to changes once it has been loaded. So this
> leaves us with a selection of non-very-attractive options, IMO. I.e.,
> we would have to:
> 
> - have to block any modifications to the hardware config that would
>   change the metadata format; this will probably result in irate users
> 
> - require XDP programs to deal with all possible metadata permutations
>   supported by that driver (by exporting them all via a BTF union or
>   similar); this means a potential for combinatorial explosion of config
>   options and as NICs become programmable themselves I'm not even sure
>   if it's possible for the driver to know ahead of time
> 
> - throw up our hands and just let the user deal with it (i.e., to
>   nothing and so require XDP programs to be reloaded if the NIC config
>   changes); this is not very friendly and is likely to lead to subtle
>   bugs if an XDP program parses the metadata assuming it is in a
>   different format than it is
> 
> Given that hardware config changes are not just done by ethtool, but
> also by things like running `tcpdump -j`, I really think we have to
> assume that they can be quite dynamic; which IMO means we have to solve
> this as part of the initial design. And I have a hard time seeing how
> this is possible without involving the kernel somehow.
> 
> Unless I'm missing something? WDYT?

First of all, thank you all guys for such a huge feedback. The last
proposal about CO-RE is like a game changer.
BTW, I've submitted a Workshop to Netdev 0x15 so we could discuss
these topics (Hints etc.) in a realtime fashion if anyone is
interested.

Second: there really are LOTS of usecases for this, not only cpumap/
veth, and not only for Rx fields, but also to be able to specify Tx
bits for XDP_TX/XDP_REDIRECT actions (like: TCP csum offload, Tx
timestamp offload like Jesper mentioned etc.).

I'm writing it on this particular line because it's one of the
majors I wanted to clarify as Toke does.

> -Toke

Thanks,
Al