Challenges of XDP hints to overcome

[Tom Herbert <tom@herbertland.com>]

I believe there are some major challenges to overcome if XDP hints is
to become ubiquitous:

1. The amount of meta information that can be practically conveyed per
packet over the PCIe bus is limited. For instance there is no
practical way to convey a complete set of metadata for a packet like
we get on output from flow dissector with killing PCIe bandwidth. It
seems like the best we can do is maybe convey a few items such as L3,
L4 offsets. This roadblock is going to be hard to overcome without an
architectural change (CXL for instance).

2. The stack needs to be able to make use of the hints. For instance,
in Transport XDP, XDP layer could conceptually perform all the
stateless processing for a TCP/IP packet and jump into TCP code at the
point where connection lookup is done. This is feasible to do in
software XDP, presumably some explicit program like in eBPF would need
to be offloaded to make it work on hardware

3. Device is a black box to the stack. We really don't want to get
into another situation like protocol specific checksum offloads where
the device is doing something that may or may not be doing in a way
the stack thinks it's appropriate. We need full transparency as to
what the device is doing. IMO, a requirement of XDP hints should be
that the device MUST be user programmable and the program for parsing
and metadata extraction is under control and visible to the stack.
Effectively, this extends the stack into the device (Split Plane
Acceleration). More specifically, to eliminate any possibility of
ambiguity and achieve "write once, run anywhere", we want the *exact*
same program to be run by the stack in software and then is offloaded
to the hardware-- this is the model of the PANDA Parser.

Tom