From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mail.toke.dk (Postfix) with ESMTPS id C0E679A5A20 for ; Thu, 29 Sep 2022 04:14:19 +0200 (CEST) Authentication-Results: mail.toke.dk; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=GysY8Si+ DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664417658; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=eCW9KL3vBxfDg7HvobNkW0lqNA/HJiSrx6HqI8xEZ/s=; b=GysY8Si+b8qt0iQERgtYwCEJkd/dhOV61sgK7Z5HacoNwI4BFVD4/b+WIN0SYDWa+3fuUK kgN+CcS9LFddK3ZRFSTTJiBrNMSAYO7XANJiNRNIJESYwJpCDAITw528UsIrQraHnJ6x3Q 9AVV6UyY/fx3VdFlmDspA83c5UVzBvA= Received: from mail-oa1-f70.google.com (mail-oa1-f70.google.com [209.85.160.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-665-y1QlDqv2OnKJtZdJxwoorw-1; Wed, 28 Sep 2022 22:14:17 -0400 X-MC-Unique: y1QlDqv2OnKJtZdJxwoorw-1 Received: by mail-oa1-f70.google.com with SMTP id 586e51a60fabf-12d265203aeso120799fac.2 for ; Wed, 28 Sep 2022 19:14:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=eCW9KL3vBxfDg7HvobNkW0lqNA/HJiSrx6HqI8xEZ/s=; b=wS5ByicnaVoMLqlxfbKKJOAQTWNWBUDwYFpaWhvPcHnMBwWROOWw4VHE9fy5a5uAuI d+7FYWKewXJ4r4BIjmjUdzszZ3QxnRpxCUFmp8JPq75Q89KGh8qXMn8M1HvZNLvrMTJG Dj1zdHxRSH7qqrlNG0pqL84Om831iErkU0M6Cuw5vG4EsOOwz5oZomOqse45iqKQVqY8 3eTQ0qguSgtsB6Yg1uenivqgDOXYwFsTVviY0EDC1cs1oSO1bTNKx/KuooojnsUKj4q0 +80itLnbrnw6TrVHD4vDcWpG9sNhyny5+GkYpMhaN37W7cMpnK1VknF/S0tOgtGyMYyS mA0g== X-Gm-Message-State: ACrzQf2aQg5Apt9KGhiZ/7zlTsiyLUrG9/0hoSLk/Hi/OfjcvF4EvqNE 5KxAKooELFiLZ37NIZbB5AUtZ2qt+bTFNFfR2vsqEkXy5ZFczta1pK4cik6D/zSPebwo3ScDRLc PrXBzo/lahOYevv7SqZwUQEXmtB7x25c= X-Received: by 2002:a05:6830:1550:b0:659:68f7:492b with SMTP id l16-20020a056830155000b0065968f7492bmr379970otp.237.1664417654099; Wed, 28 Sep 2022 19:14:14 -0700 (PDT) X-Google-Smtp-Source: AMsMyM5GbfSJ2XVlE6X7Lig+fPX5Rf96SErdXfELtPSoJlmowC1y0nS9+7xdo/klemPzTzv/JMOqckmVWatCuF7+nqM= X-Received: by 2002:a05:6830:1550:b0:659:68f7:492b with SMTP id l16-20020a056830155000b0065968f7492bmr379954otp.237.1664417653684; Wed, 28 Sep 2022 19:14:13 -0700 (PDT) MIME-Version: 1.0 References: <30dd392b-ff2f-1e25-175a-6666dde200ed@hetzner-cloud.de> <2d2d13bd-da93-2e04-3fd1-a5fd4713d7df@kernel.org> In-Reply-To: From: Jason Wang Date: Thu, 29 Sep 2022 10:14:02 +0800 Message-ID: To: Lorenzo Bianconi X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-MailFrom: jasowang@redhat.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation Message-ID-Hash: MPDYJJQLOBL22EAQRU7DSL7EO3YWQQAA X-Message-ID-Hash: MPDYJJQLOBL22EAQRU7DSL7EO3YWQQAA X-Mailman-Approved-At: Thu, 29 Sep 2022 12:45:35 +0200 CC: David Ahern , Jesper Dangaard Brouer , Marcus Wichelmann , xdp-newbies@vger.kernel.org, cloud@xdp-project.net, brouer@redhat.com, Anton Protopopov , Kumar Kartikeya Dwivedi , "Karlsson, Magnus" X-Mailman-Version: 3.3.5 Precedence: list Subject: [xdp-cloud] Re: Questions about Offloads and XDP-Hints regarding a Cloud-Provider Use-Case List-Id: XDP in the cloud Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Thu, Sep 29, 2022 at 6:37 AM Lorenzo Bianconi wrote: > > > On 9/28/22 11:07 AM, Jesper Dangaard Brouer wrote: > > > > > > On 28/09/2022 15.54, Marcus Wichelmann wrote: > > >> > > >> I'm working for a cloud hosting provider and we're working on a new > > >> XDP-based networking stack for our VM-Hosts that uses XDP to > > >> accelerate the connectivity of our qemu/KVM VMs to the outside. > > >> > > > > > > Welcome to the community! Sounds like an excellent use-case and > > > opportunity for speeding up the RX packets from physical NIC into the > > > VM. Good to hear someone (again) having this use-case. I've personally > > > > +1 +1. This is somehow what I wanted to achieve in the past. > > > > > not been focused on this use-case lately, mostly because community > > > members that I was interacting with changed jobs, away from cloud > > > hosting companies. Good to have a user back in this area! > > > > > > > > >> For this, we use XDP_REDIRECT to forward packets between the physical > > >> host NIC and the VM tap devices. The main issue we have now is, that > > >> our VM guests have some virtio NIC offloads enabled: rx/tx > > >> checksumming, TSO/GSO, GRO and Scatter-Gather. > > > > > > Supporting RX-checksumming is part of the plans for XDP-hints, although > > > virtio_net is not part of my initial patchset. > > > > Lorenzo and I both had versions of a patch to propagate rx csum > > validation to the VM on xdp redirect. I do not recall a version after > > this one: > > > > https://lore.kernel.org/netdev/cover.1622222367.git.lorenzo@kernel.org/ > > > > and I lost of track of what change is needed for it to go in. > > iirc the blocking point was the missing CHECKSUM_COMPLETE support (we support > just CHECKSUM_UNNECESSARY). We can conver it with XDP hw-hints. How about other metadata like gso_type, gso_segs, csum_start and csum_offset etc? This looks like a must for GSO to work even if multi-buffer is supported. Thanks > > Regards, > Lorenzo > > > > > > > > > XDP-redirect with GRO and Scatter-Gather frames are part of the > > > multi-buff effort (Cc Lorenzo), but currently XDP_REDIRECT with > > > multi-buff is disabled (except for cpumap), because the lack of > > > XDP-feature bits, meaning we cannot determine (in kernel) if receiving > > > net_device supports multi-buff (Cc Kumar). > > > > > >> Currently, these offloads (especially TSO/GSO) are incompatible with > > >> XDP_REDIRECT and result in packets being dropped. Because disabling > > >> these offloads in all our customer VMs is not a good option, we're > > >> searching for ways to support these offloads with XDP. > > >> > > > > > > To David Ahern, didn't the kernel recently loosen up on having to > > > disable these offloads for KVM virtio_net? > > > > not that I am aware. Still need tx offloads disabled. > > > > > > This summarizes what I was looking into back in 2020, along with the > > current state of XDP for VM use case: > > > > https://legacy.netdevconf.info/0x14/pub/slides/24/netdev-0x14-XDP-and-the-cloud.pdf > > > > source code is still on github too. > >