From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by mail.toke.dk (Postfix) with ESMTPS id 2254D9C52DF for ; Wed, 23 Nov 2022 20:14:35 +0100 (CET) Authentication-Results: mail.toke.dk; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=FQ3vf9Yt Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 9B7CBB81F2A; Wed, 23 Nov 2022 19:14:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 929C3C433C1; Wed, 23 Nov 2022 19:14:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669230873; bh=KbrG34cmabGhC68gT1YoUcErf4n0xzOwCvNVvok0PiI=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=FQ3vf9Yt/Tqn291D7wwpELSG8JKOA34K8tluIpaiJn5Mqb2sX9jb7Dy0WZmPtFAm9 ggumioiygzhDro/VU74pJOJInsUJXa9LeTDt25ipb92OLGeEQxZIL8tJm021oclbS+ +iT7d9OCNwS5e4UuNYq/IEoo0IXSjaQIwr4RLQT+HXwzE/jTNU/PS1grcY6NJ5/Zxi YpBf2YB1cZHkwtl7wwP3TzNVknTtC6dtsvWDnAnRNy9UYnhdq7QhUTZq/7NqCPXHWN q+/EX7kviPpHFA7vXjHdGM/6I4/WHLQpSuF27YqxpdtOSq+LDxbGR5HVQH/n9FZwcQ inkC8LYvJUpVg== Date: Wed, 23 Nov 2022 11:14:31 -0800 From: Jakub Kicinski To: Stanislav Fomichev Message-ID: <20221123111431.7b54668e@kernel.org> In-Reply-To: References: <20221121182552.2152891-1-sdf@google.com> <20221121182552.2152891-7-sdf@google.com> <874jupviyc.fsf@toke.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-ID-Hash: QQCOVN5WODHROUW6ZGDZI5CXUDIC2JTX X-Message-ID-Hash: QQCOVN5WODHROUW6ZGDZI5CXUDIC2JTX X-MailFrom: kuba@kernel.org X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Toke =?UTF-8?B?SMO4aWxhbmQtSsO4cmdlbnNlbg==?= , bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, haoluo@google.com, jolsa@kernel.org, Tariq Toukan , David Ahern , Willem de Bruijn , Jesper Dangaard Brouer , Anatoly Burakov , Alexander Lobakin , Magnus Karlsson , Maryam Tahhan , xdp-hints@xdp-project.net, netdev@vger.kernel.org X-Mailman-Version: 3.3.7 Precedence: list Subject: [xdp-hints] Re: [PATCH bpf-next v2 6/8] mlx4: Introduce mlx4_xdp_buff wrapper for xdp_buff List-Id: XDP hardware hints design discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, 23 Nov 2022 10:26:41 -0800 Stanislav Fomichev wrote: > > This embedding trick works for drivers that put xdp_buff on the stack, > > but mlx5 supports XSK zerocopy, which uses the xsk_buff_pool for > > allocating them. This makes it a bit awkward to do the same thing there; > > and since it's probably going to be fairly common to do something like > > this, how about we just add a 'void *drv_priv' pointer to struct > > xdp_buff that the drivers can use? The xdp_buff already takes up a full > > cache line anyway, so any data stuffed after it will spill over to a new > > one; so I don't think there's much difference performance-wise. > > I guess the alternative is to extend xsk_buff_pool with some new > argument for xdp_buff tailroom? (so it can kmalloc(sizeof(xdp_buff) + > xdp_buff_tailroom)) > But it seems messy because there is no way of knowing what the target > device's tailroom is, so it has to be a user setting :-/ > I've started with a priv pointer in xdp_buff initially, it seems fine > to go back. I'll probably convert veth/mlx4 to the same mode as well > to avoid having different approaches in different places.. Can we not do this please? Add 16B of "private driver space" after the xdp_buff in xdp_buff_xsk (we have 16B to full cacheline), the drivers decide how they use it. Drivers can do BUILD_BUG_ON() for their expected size and cast that to whatever struct they want. This is how various offloads work, the variable size tailroom would be an over design IMO. And this way non XSK paths can keep its normal typing. > > I'll send my patch to add support to mlx5 (using the drv_priv pointer > > approach) separately. > > Saw them, thanks! Will include them in v3+.