XDP hardware hints discussion mail archive
 help / color / mirror / Atom feed
From: Jesper Dangaard Brouer <jbrouer@redhat.com>
To: Paul Menzel <pmenzel@molgen.mpg.de>,
	Alexander Lobakin <alexandr.lobakin@intel.com>
Cc: brouer@redhat.com, bpf@vger.kernel.org,
	xdp-hints@xdp-project.net, martin.lau@kernel.org,
	daniel@iogearbox.net, netdev@vger.kernel.org, ast@kernel.org,
	Stanislav Fomichev <sdf@google.com>,
	yoong.siang.song@intel.com, anthony.l.nguyen@intel.com,
	intel-wired-lan@lists.osuosl.org
Subject: [xdp-hints] Re: [Intel-wired-lan] [PATCH bpf-next V1] igc: enable and fix RX hash usage by netstack
Date: Thu, 16 Feb 2023 14:29:43 +0100	[thread overview]
Message-ID: <88e5c6b2-8acc-6585-100d-7b62320e5555@redhat.com> (raw)
In-Reply-To: <6a5ded96-2425-ff9b-c1b1-eca1c103164c@molgen.mpg.de>


On 14/02/2023 16.00, Paul Menzel wrote:
> 
> Thank you very much for your patch.

Thanks for your review :-)

> Am 10.02.23 um 16:07 schrieb Jesper Dangaard Brouer:
>> When function igc_rx_hash() was introduced in v4.20 via commit 
>> 0507ef8a0372
>> ("igc: Add transmit and receive fastpath and interrupt handlers"), the
>> hardware wasn't configured to provide RSS hash, thus it made sense to not
>> enable net_device NETIF_F_RXHASH feature bit.
>>
>> The NIC hardware was configured to enable RSS hash info in v5.2 via 
>> commit
>> 2121c2712f82 ("igc: Add multiple receive queues control supporting"), but
>> forgot to set the NETIF_F_RXHASH feature bit.
>>
>> The original implementation of igc_rx_hash() didn't extract the associated
>> pkt_hash_type, but statically set PKT_HASH_TYPE_L3. The largest portions of
>> this patch are about extracting the RSS Type from the hardware and mapping
>> this to enum pkt_hash_types. This were based on Foxville i225 software 
>> user
> 
> s/This were/This was/

Fixed for V2

>> manual rev-1.3.1 and tested on Intel Ethernet Controller I225-LM (rev 
>> 03).
>>
>> For UDP it's worth noting that RSS (type) hashing have been disabled both for
>> IPv4 and IPv6 (see IGC_MRQC_RSS_FIELD_IPV4_UDP + IGC_MRQC_RSS_FIELD_IPV6_UDP)
>> because hardware RSS doesn't handle fragmented pkts well when enabled 
>> (can cause out-of-order). This result in PKT_HASH_TYPE_L3 for UDP packets, and
> 
> result*s*

Fixed for V2

> 
>> hash value doesn't include UDP port numbers. Not being PKT_HASH_TYPE_L4, have
>> the effect that netstack will do a software based hash calc calling into
>> flow_dissect, but only when code calls skb_get_hash(), which doesn't
>> necessary happen for local delivery.
> 
> Excuse my ignorance, but is that bug visible in practice by users 
> (performance?) or is that fix needed for future work?
> 
>> Fixes: 2121c2712f82 ("igc: Add multiple receive queues control supporting")
>> Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
>> ---
>>   drivers/net/ethernet/intel/igc/igc.h      |   52 
>> +++++++++++++++++++++++++++++
>>   drivers/net/ethernet/intel/igc/igc_main.c |   35 +++++++++++++++++---
>>   2 files changed, 83 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/intel/igc/igc.h 
>> b/drivers/net/ethernet/intel/igc/igc.h
>> index df3e26c0cf01..a112eeb59525 100644
>> --- a/drivers/net/ethernet/intel/igc/igc.h
>> +++ b/drivers/net/ethernet/intel/igc/igc.h
>> @@ -311,6 +311,58 @@ extern char igc_driver_name[];
>>   #define IGC_MRQC_RSS_FIELD_IPV4_UDP    0x00400000
>>   #define IGC_MRQC_RSS_FIELD_IPV6_UDP    0x00800000
>> +/* RX-desc Write-Back format RSS Type's */
>> +enum igc_rss_type_num {
>> +    IGC_RSS_TYPE_NO_HASH        = 0,
>> +    IGC_RSS_TYPE_HASH_TCP_IPV4    = 1,
>> +    IGC_RSS_TYPE_HASH_IPV4        = 2,
>> +    IGC_RSS_TYPE_HASH_TCP_IPV6    = 3,
>> +    IGC_RSS_TYPE_HASH_IPV6_EX    = 4,
>> +    IGC_RSS_TYPE_HASH_IPV6        = 5,
>> +    IGC_RSS_TYPE_HASH_TCP_IPV6_EX    = 6,
>> +    IGC_RSS_TYPE_HASH_UDP_IPV4    = 7,
>> +    IGC_RSS_TYPE_HASH_UDP_IPV6    = 8,
>> +    IGC_RSS_TYPE_HASH_UDP_IPV6_EX    = 9,
>> +    IGC_RSS_TYPE_MAX        = 10,
>> +};
>> +#define IGC_RSS_TYPE_MAX_TABLE        16
>> +#define IGC_RSS_TYPE_MASK        0xF
>> +
>> +/* igc_rss_type - Rx descriptor RSS type field */
>> +static inline u8 igc_rss_type(union igc_adv_rx_desc *rx_desc)
>> +{
>> +    /* RSS Type 4-bit number: 0-9 (above 9 is reserved) */
>> +    return rx_desc->wb.lower.lo_dword.hs_rss.pkt_info & IGC_RSS_TYPE_MASK;
>> +}
> 
> Is it necessary to specficy the length of the return value, or could it 
> be `unsigned int`. Using “native” types is normally more performant [1]. 
> `scripts/bloat-o-meter` might help to verify that.
> 

Thanks for the link[1].
Alex/Olek also pointed this out.

The Agner's instruction latency tables[2] do indicate the latency is
slightly higher for r8 and r16 (and m8/m16).  And we likely need to look 
at the zero-extend variants movzx.

I think we should investigate this with "tool" godbolt.org as
scripts/bloat-o-meter will only tell us about code size.
I will experiment a bit and report back :-)

[2] https://www.agner.org/optimize/instruction_tables.pdf

> […]
> 
>>   static inline void igc_rx_hash(struct igc_ring *ring,
>>                      union igc_adv_rx_desc *rx_desc,
>>                      struct sk_buff *skb)
>>   {
>> -    if (ring->netdev->features & NETIF_F_RXHASH)
>> -        skb_set_hash(skb,
>> -                 le32_to_cpu(rx_desc->wb.lower.hi_dword.rss),
>> -                 PKT_HASH_TYPE_L3);
>> +    if (ring->netdev->features & NETIF_F_RXHASH) {
>> +        u32 rss_hash = le32_to_cpu(rx_desc->wb.lower.hi_dword.rss);
>> +        u8  rss_type = igc_rss_type(rx_desc);
> 
> Amongst others, also here.

Do notice I expect compiler to optimize this, such that is doesn't place 
this variable on the stack.

>> +        enum pkt_hash_types hash_type;
>> +
>> +        hash_type = igc_rss_type_table[rss_type].hash_type;
>> +        skb_set_hash(skb, rss_hash, hash_type);
>> +    }
>>   }
>>   static void igc_rx_vlan(struct igc_ring *rx_ring,
>> @@ -6501,6 +6527,7 @@ static int igc_probe(struct pci_dev *pdev,
>>       netdev->features |= NETIF_F_TSO;
>>       netdev->features |= NETIF_F_TSO6;
>>       netdev->features |= NETIF_F_TSO_ECN;
>> +    netdev->features |= NETIF_F_RXHASH;
>>       netdev->features |= NETIF_F_RXCSUM;
>>       netdev->features |= NETIF_F_HW_CSUM;
>>       netdev->features |= NETIF_F_SCTP_CRC;
> 
> 
> Kind regards,
> 
> Paul
> 
> 
> [1]: https://notabs.org/coding/smallIntsBigPenalty.htm
> 


  parent reply	other threads:[~2023-02-16 13:29 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-02-10 15:07 [xdp-hints] " Jesper Dangaard Brouer
2023-02-10 15:23 ` [xdp-hints] " Jesper Dangaard Brouer
2023-02-14 13:21 ` Alexander Lobakin
2023-02-16 16:46   ` Jesper Dangaard Brouer
2023-02-20 15:39     ` Alexander Lobakin
2023-02-22 15:00       ` Jesper Dangaard Brouer
2023-02-24 16:41         ` Alexander Lobakin
2023-02-27 14:24           ` Alexander Lobakin
2023-02-14 15:00 ` [xdp-hints] Re: [Intel-wired-lan] " Paul Menzel
2023-02-14 15:13   ` Alexander Lobakin
2023-02-16 15:17     ` Jesper Dangaard Brouer
2023-02-16 15:43       ` Alexander Lobakin
2023-02-27 14:53         ` Alexander Lobakin
2023-02-16 13:29   ` Jesper Dangaard Brouer [this message]
2023-02-16 15:34     ` Alexander Lobakin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://lists.xdp-project.net/postorius/lists/xdp-hints.xdp-project.net/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=88e5c6b2-8acc-6585-100d-7b62320e5555@redhat.com \
    --to=jbrouer@redhat.com \
    --cc=alexandr.lobakin@intel.com \
    --cc=anthony.l.nguyen@intel.com \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=daniel@iogearbox.net \
    --cc=intel-wired-lan@lists.osuosl.org \
    --cc=martin.lau@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pmenzel@molgen.mpg.de \
    --cc=sdf@google.com \
    --cc=xdp-hints@xdp-project.net \
    --cc=yoong.siang.song@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox