From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mail.toke.dk (Postfix) with ESMTPS id 228589C5D7D for ; Fri, 25 Nov 2022 01:36:57 +0100 (CET) Authentication-Results: mail.toke.dk; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=BcgnPUVn DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1669336616; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WboZf6G6Aqw+I/RSdsOOvbXEHQPl8DYBko3EUjwbRNg=; b=BcgnPUVne5xHeNSakUR4XEwz2vX8caPW4dqnCeLn6zN9hteQ3YjUq+8GU4W/EKp0h9Qu4b yjyWPqNPOVk9Ghm0BtZlzpqARu4J/fcHrEg9O4NAvRGBtmJD3ij4DopDLWrWxmf+3dN1HH DnOxVxXW9e7EHuXC8cn4DknQu1XlyQ0= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-633-3vpv2qXPNbO1uFlOvdKcyg-1; Thu, 24 Nov 2022 19:36:55 -0500 X-MC-Unique: 3vpv2qXPNbO1uFlOvdKcyg-1 Received: by mail-ed1-f69.google.com with SMTP id m13-20020a056402510d00b0046913fa9291so1655758edd.6 for ; Thu, 24 Nov 2022 16:36:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bVrOkDrmMGpa5h7AIlaFraeH3E0rHqCXhdxo/iWpc3M=; b=KwnVclvq5wvJFg4Gg7eMMb3hhUbtp7HPnRd+DfB4F5lO5Rmq6bSsN171vFGaVgA1yV qD8CGw1RO7s2R4y+Uwlh/0UjaOhzB5q2b76AdrEN4+a3P3SkTFTzHlJxyDOoJKrACt3+ pOONwmqxqdaplzywo6s/OTez4aP5+JgUFyT/pqScbW04dIksHFkbtHAXwCXZcKXyPpKJ YfHByAe/muQgdV/n2c+BwaRBkPSJAlqC93IgiPgBAbRYl2xCahY4tm95DaDuzwbYBn0Z rTLxKqNVMviG+TwoIZaIE6ZY8EHY2pOzenrGIGXydV9G6MeAogUuF9Bl11OOPuT6rQtq z7vg== X-Gm-Message-State: ANoB5pmMK5ENJcgeV4PfIfD66BPdC8JBuiBv5+oDXWLbMx3mdWNi3VDO GYfWxMhq0mOso/iihEJYjMSoNUa5xt+r+WMwtybxQQlw++7UhmhVonXMJ5MAGv8/q8sV4yHg/nX iHUOK5pIhabRBb9lVU5Ch X-Received: by 2002:a17:906:810:b0:7ba:5b85:994f with SMTP id e16-20020a170906081000b007ba5b85994fmr7093354ejd.510.1669336613670; Thu, 24 Nov 2022 16:36:53 -0800 (PST) X-Google-Smtp-Source: AA0mqf5kN53QWAHI3CajHtXDJoe0dz9z+qfwYwqCLE3CqwTYapDEN2akUb+MJf0gMCXGfUJU4PLHlQ== X-Received: by 2002:a17:906:810:b0:7ba:5b85:994f with SMTP id e16-20020a170906081000b007ba5b85994fmr7093337ejd.510.1669336613264; Thu, 24 Nov 2022 16:36:53 -0800 (PST) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id i10-20020a05640242ca00b00467c3cbab6fsm1115299edc.77.2022.11.24.16.36.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Nov 2022 16:36:52 -0800 (PST) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id D11337EB6D8; Fri, 25 Nov 2022 01:36:51 +0100 (CET) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= To: Maciej Fijalkowski In-Reply-To: References: <20221121182552.2152891-1-sdf@google.com> <20221121182552.2152891-7-sdf@google.com> <874jupviyc.fsf@toke.dk> <20221123111431.7b54668e@kernel.org> <871qptuyie.fsf@toke.dk> <20221123174746.418920e5@kernel.org> <87edts2z8n.fsf@toke.dk> X-Clacks-Overhead: GNU Terry Pratchett Date: Fri, 25 Nov 2022 01:36:51 +0100 Message-ID: <874jun3m58.fsf@toke.dk> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Message-ID-Hash: IIMXRNY75GGVVHGOTRRJWOIHT2XYFCAB X-Message-ID-Hash: IIMXRNY75GGVVHGOTRRJWOIHT2XYFCAB X-MailFrom: toke@redhat.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Jakub Kicinski , sdf@google.com, bpf@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, haoluo@google.com, jolsa@kernel.org, Tariq Toukan , David Ahern , Willem de Bruijn , Jesper Dangaard Brouer , Anatoly Burakov , Alexander Lobakin , Magnus Karlsson , Maryam Tahhan , xdp-hints@xdp-project.net, netdev@vger.kernel.org X-Mailman-Version: 3.3.7 Precedence: list Subject: [xdp-hints] Re: [PATCH bpf-next v2 6/8] mlx4: Introduce mlx4_xdp_buff wrapper for xdp_buff List-Id: XDP hardware hints design discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Maciej Fijalkowski writes: > On Thu, Nov 24, 2022 at 04:17:01PM +0100, Maciej Fijalkowski wrote: >> On Thu, Nov 24, 2022 at 03:39:20PM +0100, Toke H=C3=B8iland-J=C3=B8rgens= en wrote: >> > Jakub Kicinski writes: >> >=20 >> > > On Wed, 23 Nov 2022 22:55:21 +0100 Toke H=C3=B8iland-J=C3=B8rgensen = wrote: >> > >> > Good idea, prototyped below, lmk if it that's not what you had in= mind. >> > >> > >> > >> > struct xdp_buff_xsk { >> > >> > =09struct xdp_buff xdp; /* 0 5= 6 */ >> > >> > =09u8 cb[16]; /* 56 1= 6 */ >> > >> > =09/* --- cacheline 1 boundary (64 bytes) was 8 bytes ago --- */ = =20 >> > >>=20 >> > >> As pahole helpfully says here, xdp_buff is actually only 8 bytes fr= om >> > >> being a full cache line. I thought about adding a 'cb' field like t= his >> > >> to xdp_buff itself, but figured that since there's only room for a >> > >> single pointer, why not just add that and let the driver point it t= o >> > >> where it wants to store the extra context data? >> > > >> > > What if the driver wants to store multiple pointers or an integer or >> > > whatever else? The single pointer seems quite arbitrary and not >> > > strictly necessary. >> >=20 >> > Well, then you allocate a separate struct and point to that? Like I di= d >> > in mlx5: >> >=20 >> >=20 >> > +=09struct mlx5_xdp_ctx mlctx =3D { .cqe =3D cqe, .rq =3D rq }; >> > +=09struct xdp_buff xdp =3D { .drv_priv =3D &mlctx }; >> >=20 >> > but yeah, this does give an extra pointer deref on access. I'm not >> > really opposed to the cb field either, I just think it's a bit odd to >> > put it in struct xdp_buff_xsk; that basically requires the driver to >> > keep the layouts in sync. >> >=20 >> > Instead, why not but a cb field into xdp_buff itself so it can be used >> > for both the XSK and the non-XSK paths? Then the driver can just >> > typecast the xdp_buff into its own struct that has whatever data it >> > wants in place of the cb field? >>=20 >> Why can't you simply have a pointer to xdp_buff in driver specific >> xdp_buff container which would point to xdp_buff that is stack based (or >> whatever else memory that will back it up - I am about to push a change >> that makes ice driver embed xdp_buff within a struct that represents Rx >> ring) for XDP path and for ZC the pointer to xdp_buff that you get from >> xsk_buff_pool ? This would satisfy both sides I believe and would let us >> keep the same container struct. >>=20 >> struct mlx4_xdp_buff { >> =09struct xdp_buff *xdp; >> =09struct mlx4_cqe *cqe; >> =09struct mlx4_en_dev *mdev; >> =09struct mlx4_en_rx_ring *ring; >> =09struct net_device *dev; >> }; > > Nah this won't work from kfunc POV, probably no way to retrieve the > mlx4_xdp_buff based on xdp_buff ptr that needs to be used as an arg. > > Sorry I'll think more about it, in the meantime let's hear more voices > whether we should keep Stan's original approach + modify xdp_buff_xsk or > go with Toke's proposal. OK, so I played around with the mlx5 code a bit more, and I think the "wrapping struct + cb area" can be made to work without too many ugly casts; I'll send an updated version of the mlx5 patches with this incorporated tomorrow, after I've run some tests... -Toke