From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from flow8-smtp.messagingengine.com (flow8-smtp.messagingengine.com [103.168.172.143]) by mail.toke.dk (Postfix) with ESMTPS id 3DDF3A7E8C1 for ; Wed, 07 Aug 2024 22:38:25 +0200 (CEST) Authentication-Results: mail.toke.dk; dkim=pass (2048-bit key; unprotected) header.d=dxuuu.xyz header.i=@dxuuu.xyz header.a=rsa-sha256 header.s=fm2 header.b=E+eKG3g1; dkim=pass (2048-bit key; unprotected) header.d=messagingengine.com header.i=@messagingengine.com header.a=rsa-sha256 header.s=fm3 header.b=b96naOm6 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailflow.nyi.internal (Postfix) with ESMTP id 6FC3B200CD6; Wed, 7 Aug 2024 16:38:24 -0400 (EDT) Received: from wimap24 ([10.202.2.84]) by compute3.internal (MEProxy); Wed, 07 Aug 2024 16:38:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dxuuu.xyz; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1723063104; x=1723070304; bh=jyuTeJGDcuQF3lzGu4CM7mjVX08aa1yYpmFdb0KuMkg=; b= E+eKG3g18OxvHvRzsf6ur2aTpQZ9AzI8YfWphmPd0cP/woMQ4Y3QVK1CWtwyfC0y cHTeaFxhMH1uSN585hjjDqb87LprAEz9VK+9wE/aaFBd7xWr105XIJW2HiO3CCBT YUhe5sKACQ2+4qxt6XjnpRHJuImN2bklW8zmfcEAFP7Lou43NRM5qKq7IDAugdai IYwG14dKzFFps6YPHSCr8q3rMVjgC39MnFUcYuPCpyq6euXGEsVrw9WBPuceev/e X1NgaLyTbyhKd84J12Ahg8rIJO1jqyZ0gWDxM/DU4rzl1Ec33URQG7nXdRjdZ8vH rYKBskImd2bUqczPtF0xdQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1723063104; x= 1723070304; bh=jyuTeJGDcuQF3lzGu4CM7mjVX08aa1yYpmFdb0KuMkg=; b=b 96naOm6Tmp/fSG/qTnRg0+F8M1bkR3E6CbDhH/DghMASO9cQQZXdb+LxscNKRR+9 7Fk34G4TL+jkTV6cjuk1odFzQ1NlafI85IkaEBBVXbJELeIamzWo5vQoPavmZTfY EGzqi+M7c6MwPerN2+6XDk2A51wIsc06LLhESfT3n6f9myyWLNa39joOmjaiSdLt SzS10PV4vNew1tgyh0/ihr7Ntoyai8cLMI1B3uhwDG/7G0ganJJiQD826YGpf2KG 2ihMc9VIedRkjSJ8h41pV7ucNf0knq9CR0h2UorMPi98cUM3wsY95XiAAvnhtdyQ Yxn+V/lEX1dz2j6fa+wjQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeeftddrledtgdduheduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne gfrhhlucfvnfffucdlfeehmdenucfjughrpefoggffhffvvefkjghfufgtgfesthejredt redttdenucfhrhhomhepfdffrghnihgvlhcuighufdcuoegugihusegugihuuhhurdighi iiqeenucggtffrrghtthgvrhhnpeegleeifffhudduueekhfeifefgffegudelveejfeff ueekgfdtledvvdeffeeiudenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmh grihhlfhhrohhmpegugihusegugihuuhhurdighiiipdhnsggprhgtphhtthhopedt X-ME-Proxy: Feedback-ID: i6a694271:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 9567E25E006A; Wed, 7 Aug 2024 16:38:23 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface MIME-Version: 1.0 Date: Wed, 07 Aug 2024 13:38:03 -0700 From: "Daniel Xu" To: "Alexander Lobakin" , "Alexei Starovoitov" , "Daniel Borkmann" , "Andrii Nakryiko" Message-Id: In-Reply-To: <20220628194812.1453059-33-alexandr.lobakin@intel.com> References: <20220628194812.1453059-1-alexandr.lobakin@intel.com> <20220628194812.1453059-33-alexandr.lobakin@intel.com> Content-Type: text/plain Content-Transfer-Encoding: 7bit Message-ID-Hash: BRLYSTVVBZKNUGR5FILJMV2FHPH2JT7Q X-Message-ID-Hash: BRLYSTVVBZKNUGR5FILJMV2FHPH2JT7Q X-MailFrom: dxu@dxuuu.xyz X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Larysa Zaremba , Michal Swiatkowski , Jesper Dangaard Brouer , =?UTF-8?Q?Bj=C3=B6rn_T=C3=B6pel?= , Magnus Karlsson , Maciej Fijalkowski , Jonathan Lemon , "toke@redhat.com" , Lorenzo Bianconi , David Miller , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Jesse Brandeburg , John Fastabend , Yajun Deng , Willem de Bruijn , "bpf@vger.kernel.org" , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, xdp-hints@xdp-project.net X-Mailman-Version: 3.3.9 Precedence: list Subject: [xdp-hints] Re: [PATCH RFC bpf-next 32/52] bpf, cpumap: switch to GRO from netif_receive_skb_list() List-Id: XDP hardware hints design discussion Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Hi Alexander, On Tue, Jun 28, 2022, at 12:47 PM, Alexander Lobakin wrote: > cpumap has its own BH context based on kthread. It has a sane batch > size of 8 frames per one cycle. > GRO can be used on its own, adjust cpumap calls to the > upper stack to use GRO API instead of netif_receive_skb_list() which > processes skbs by batches, but doesn't involve GRO layer at all. > It is most beneficial when a NIC which frame come from is XDP > generic metadata-enabled, but in plenty of tests GRO performs better > than listed receiving even given that it has to calculate full frame > checksums on CPU. > As GRO passes the skbs to the upper stack in the batches of > @gro_normal_batch, i.e. 8 by default, and @skb->dev point to the > device where the frame comes from, it is enough to disable GRO > netdev feature on it to completely restore the original behaviour: > untouched frames will be being bulked and passed to the upper stack > by 8, as it was with netif_receive_skb_list(). > > Signed-off-by: Alexander Lobakin > --- > kernel/bpf/cpumap.c | 43 ++++++++++++++++++++++++++++++++++++++----- > 1 file changed, 38 insertions(+), 5 deletions(-) > AFAICT the cpumap + GRO is a good standalone improvement. I think cpumap is still missing this. I have a production use case for this now. We want to do some intelligent RX steering and I think GRO would help over list-ified receive in some cases. We would prefer steer in HW (and thus get existing GRO support) but not all our NICs support it. So we need a software fallback. Are you still interested in merging the cpumap + GRO patches? Thanks, Daniel