From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-il1-x12f.google.com (mail-il1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) by mail.toke.dk (Postfix) with ESMTPS id A15D09B0239 for ; Fri, 28 Oct 2022 20:46:24 +0200 (CEST) Authentication-Results: mail.toke.dk; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20210112 header.b=OEM2S29V Received: by mail-il1-x12f.google.com with SMTP id l6so3413955ilq.3 for ; Fri, 28 Oct 2022 11:46:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=BUEnwbILm0RrJRzh9nhTtkoyV8KawPDkBr+FO7E6eJE=; b=OEM2S29VlPFvaBDNRnimgr7+Djgx9aWzQzINf/4SJFAFzJKspkkVK+HCj941UhQN3o yliKmaPXljYuNcyqLBOIZAjhEmHwPw2Pw4nkn4yJkWNwgAP8tQIqoiCK6LDyErbi+dGm a4yIvu0p/RXgIBWcbe53jaSxwmnTcdbsBNZwLExc2wVY8uCnIR5tRq+fGutFJBRbhpRA AwJvGfMs9IBszPsqi1IUXZaO4bDrX6W+wI6RzQmtpK67ZVPA2fucmMT3nft0y6g5LyRa Q8RuwYHjGlGPxQGAyS6KMkqRxq8JFxkC2U2cBtTZoZMfy/tpzX/LDgtbMxpLYu7e/qgT nPZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=BUEnwbILm0RrJRzh9nhTtkoyV8KawPDkBr+FO7E6eJE=; b=s/n4/LE/CldIUHRhd4+ZrnlzLlRL5926in15HprKxWUKVPXciygrMCpB1DNQpmb5Uy eB1K+20zDHVJnOV/LtQxXE7yo0m8vX//HvKbJDIgaUrkPGH4cLjfysqbcVJpdvbxz49M ZIJTkabRNnSWJ9jbQK5/4fqPzUfEG29YY/mEjunXofmUlexxjXXvlBERIdKLV1oADfEt JQDRCnkJEfrXvRAmXY10uaS6V4rSbw5W1YUsxUA9oGO+bJxN3hq4HwOVOxniCMp9df/6 ZniSFPiwr5hisL/Xaj1+zZd6tZ/QFjD09YSTk0b81/Wew3iSyoYXqAMlboz2NgF6SWcd cWOw== X-Gm-Message-State: ACrzQf0jhYMuMSEEIhOn8mXFedLRaJB6phdJEF8xsOnlqgu2toaP74pp OLPQf7TLcBtdo1njqmjLYW3I3RO/D8V3bcsK0DeBpw== X-Google-Smtp-Source: AMsMyM7TkTJ2M4RZSOD8R2Q4DzOTMnQMjO6dzfuwFlSN8Qvuombxc6lZqxZlnL+93s+O4iRzdw7XCcXueJVfbeUL8zw= X-Received: by 2002:a92:db03:0:b0:300:5dc4:d111 with SMTP id b3-20020a92db03000000b003005dc4d111mr365055iln.257.1666982782376; Fri, 28 Oct 2022 11:46:22 -0700 (PDT) MIME-Version: 1.0 References: <20221027200019.4106375-1-sdf@google.com> <635bfc1a7c351_256e2082f@john.notmuch> <20221028110457.0ba53d8b@kernel.org> In-Reply-To: <20221028110457.0ba53d8b@kernel.org> From: Stanislav Fomichev Date: Fri, 28 Oct 2022 11:46:11 -0700 Message-ID: To: Jakub Kicinski Content-Type: text/plain; charset="UTF-8" Message-ID-Hash: ECTOA73EMIEJSQM3ZVJE4W5F2NHTXKZH X-Message-ID-Hash: ECTOA73EMIEJSQM3ZVJE4W5F2NHTXKZH X-MailFrom: sdf@google.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: John Fastabend , bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, kpsingh@kernel.org, haoluo@google.com, jolsa@kernel.org, Willem de Bruijn , Jesper Dangaard Brouer , Anatoly Burakov , Alexander Lobakin , Magnus Karlsson , Maryam Tahhan , xdp-hints@xdp-project.net, netdev@vger.kernel.org X-Mailman-Version: 3.3.5 Precedence: list Subject: [xdp-hints] Re: [RFC bpf-next 0/5] xdp: hints via kfuncs List-Id: XDP hardware hints design discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Fri, Oct 28, 2022 at 11:05 AM Jakub Kicinski wrote: > > On Fri, 28 Oct 2022 08:58:18 -0700 John Fastabend wrote: > > A bit of extra commentary. By exposing the raw kptr to the rx > > descriptor we don't need driver writers to do anything. > > And can easily support all the drivers out the gate with simple > > one or two line changes. This pushes the interesting parts > > into userspace and then BPF writers get to do the work without > > bother driver folks and also if its not done today it doesn't > > matter because user space can come along and make it work > > later. So no scattered kernel dependencies which I really > > would like to avoid here. Its actually very painful to have > > to support clusters with N kernels and M devices if they > > have different features. Doable but annoying and much nicer > > if we just say 6.2 has support for kptr rx descriptor reading > > and all XDP drivers support it. So timestamp, rxhash work > > across the board. > > IMHO that's a bit of wishful thinking. Driver support is just a small > piece, you'll have different HW and FW versions, feature conflicts etc. > In the end kernel version is just one variable and there are many others > you'll already have to track. > > And it's actually harder to abstract away inter HW generation > differences if the user space code has to handle all of it. I've had the same concern: Until we have some userspace library that abstracts all these details, it's not really convenient to use. IIUC, with a kptr, I'd get a blob of data and I need to go through the code and see what particular type it represents for my particular device and how the data I need is represented there. There are also these "if this is device v1 -> use v1 descriptor format; if it's a v2->use this another struct; etc" complexities that we'll be pushing onto the users. With kfuncs, we put this burden on the driver developers, but I agree that the drawback here is that we actually have to wait for the implementations to catch up. Jakub mentions FW and I haven't even thought about that; so yeah, bpf programs might have to take a lot of other state into consideration when parsing the descriptors; all those details do seem like they belong to the driver code. Feel free to send it early with just a handful of drivers implemented; I'm more interested about bpf/af_xdp/user api story; if we have some nice sample/test case that shows how the metadata can be used, that might push us closer to the agreement on the best way to proceed. > > To find the offset of fields (rxhash, timestamp) you can use > > standard BTF relocations we have all this machinery built up > > already for all the other structs we read, net_devices, task > > structs, inodes, ... so its not a big hurdle at all IMO. We > > can add userspace libs if folks really care, but its just a read so > > I'm not even sure that is helpful. > > > > I think its nicer than having kfuncs that need to be written > > everywhere. My $.02 although I'll poke around with below > > some as well. Feel free to just hang tight until I have some > > code at the moment I have intel, mellanox drivers that I > > would want to support. > > I'd prefer if we left the door open for new vendors. Punting descriptor > parsing to user space will indeed result in what you just said - major > vendors are supported and that's it.