1

So I am currently working on understanding how to Receive Side Scaling (RSS) Offloads work inside DPDK (Data Plane Development Kit). So basically from my understanding depending on which offload you have selected, it calculates a Hash value and uses that to place your packet for processing on a specific Queue, which is binded to a specific CPU Core.

So I have 2 Queries regarding this:

  • When I use RSS offload of **ETH_RSS_IPV6_EX** , I am unable to get a Hash value other than Zero, meaning the packet is considered invalid in accordance to the RSS Offload selected, even though my Mellanox card supports this offload. I have sent the following Scapy packet but still hash is coming to be 0: sendp(Ether(dst="AA:AA:BB:BB:CC:DD")/IPv6(dst="a:a:a:a:a:a:a:b",src="a:a:a:a:a:a:a:c",nh=60)/IPv6ExtHdrDestOpt(nh=43,options=HAO(hoa="a:a:a:a:a:a:a:d"))/IPv6ExtHdrRouting(nh=59,type=2,addresses=["a:a:a:a:a:a:a:e"]),iface="enp4s0f0",count=1). (On which packets to manipulate I got this resource from https://learn.microsoft.com/en-us/windows-hardware/drivers/network/rss-hashing-types#ndis_hash_ipv6_ex, but I believe there might be a difference between these RSS Offloads and the ones DPDK has.) What does this EX term mean in the RSS Offload.

  • Apart from that, can someone point me to a resource that explains the RSS Offloads, one at a time, that are inside the DPDK, some of them being the following (If you have an understanding of these, providing a jest of them can also be helpful):

    #define ETH_RSS_IPV4               (1ULL << 2)
    #define ETH_RSS_FRAG_IPV4          (1ULL << 3)
    #define ETH_RSS_NONFRAG_IPV4_TCP   (1ULL << 4)
    #define ETH_RSS_NONFRAG_IPV4_UDP   (1ULL << 5)
    #define ETH_RSS_NONFRAG_IPV4_SCTP  (1ULL << 6)
    #define ETH_RSS_NONFRAG_IPV4_OTHER (1ULL << 7)
    #define ETH_RSS_IPV6               (1ULL << 8)
    #define ETH_RSS_FRAG_IPV6          (1ULL << 9)
    #define ETH_RSS_NONFRAG_IPV6_TCP   (1ULL << 10)
    #define ETH_RSS_NONFRAG_IPV6_UDP   (1ULL << 11)
    #define ETH_RSS_NONFRAG_IPV6_SCTP  (1ULL << 12)
    #define ETH_RSS_NONFRAG_IPV6_OTHER (1ULL << 13)
    #define ETH_RSS_L2_PAYLOAD         (1ULL << 14)
    #define ETH_RSS_IPV6_EX            (1ULL << 15)
    #define ETH_RSS_IPV6_TCP_EX        (1ULL << 16)
    #define ETH_RSS_IPV6_UDP_EX        (1ULL << 17)
    

Please feel free for any clarifications or elaboration you require.

Thanks in Advance.

Edit_1 (In response to Vipin's comment):

  1. I am enabling the RSS in this strucutre (ETH_RSS_IPV6_EX):

    static struct rte_eth_conf port_conf = { .rxmode = { .mq_mode = ETH_MQ_RX_RSS, .max_rx_pkt_len = RTE_ETHER_MAX_LEN, .split_hdr_size = 0, .offloads = DEV_RX_OFFLOAD_CHECKSUM, }, .rx_adv_conf = { .rss_conf = { .rss_key = NULL, .rss_hf = ETH_RSS_IPV6_EX, //ETH_RSS_IP //Orignal, //AU: Changing Hash Application }, }, .txmode = { .mq_mode = ETH_MQ_TX_NONE, }, };

  2. DPDK Version I am using is 20.08.

  3. NIC being used is Mellanox ConnectX-5

  4. I am using the DPDK l3fwd application as the base and testing on it.

Ameer Usman
  • 95
  • 10
  • share the 1. code snippet which you are using for enabling RSS. 2. DPDK version, 3. NIC under use 4. Firmware under use. Note: I am refraning from marking this question `needs improvement - missing details`. Please edit at earliest. – Vipin Varghese Nov 25 '20 at 10:44
  • Thanks @VipinVarghese , I have added the details. – Ameer Usman Nov 25 '20 at 11:04
  • thanks for the details. since I do not have the same NIC, can you tell me the result for `rss_hf = ETH_RSS_IP` also? Note: `ETH_RSS_IP` is a super set of `ETH_RSS_IPV6_EX` – Vipin Varghese Nov 25 '20 at 11:44
  • I tried it with `ETH_RSS_IP` and the above packet (scapy packet in the orignal post) calculates a hash for it (hash = 0xc0f4655c). Since also includes IPv6, I believe it just calculates hash according to that. If possible can you just use `ETH_RSS_IPV6_EX` and see what scapy packet actually calculates a hash for it. – Ameer Usman Nov 25 '20 at 12:28
  • well @AmeerUsman, if you say `ETH_RSS_IPV6` and `ETH_RSS_IPV6_EX` it can mean 2 things. 1) there might genuine issue with mxl5 (which is unlikely) 2) `ETH_RSS_IPV6_EX` is expected to be used with `ETH_RSS_IPV6` because the packet you are sending is not tcp,udp or sctp . for RSS calculation. I hope you understand the second case, where and what needs to be present in packet for RSS. I will wait for you, if not share the answer with explanation. – Vipin Varghese Nov 25 '20 at 12:34
  • I believe the problem is that there are certain fields that are not present for the `ETH_RSS_IPV6_EX` to calculate hash. So the objective is to find those fields, plug them in Scapy and send it, resulting in a hash being calculated (non-zero). I doubt there is an issue with mlx5 probably. Also there is `ETH_RSS_IPV6_TCP_EX`, in which I plugged in TCP header plus extension headers inside Scapy but still got a zero hash. – Ameer Usman Nov 25 '20 at 12:47
  • I believe you are very close to understanding. But assuming that `ETH_RSS_IPV6_EX` or `ETH_RSS_IPV6_TCP_EX` will automatically take srcip and dstip into consideration. So either experiment with scapy with various combination ipv6 packets or raise a bugzilla with mellanox if you indeed think the issue is mxl5. I will wait for your update and then update the answer and my findings later. – Vipin Varghese Nov 25 '20 at 13:18
  • @VipinVarghese, I do not think it is a bug but I cannot exactly verify this either at the moment. I have tried different combinations as well of IPv6 Extension headers but to no avail. – Ameer Usman Nov 26 '20 at 12:05
  • thank you for understanding it is not bug, with respect to combination yu have tried. Update the ticket with the combination you have tried or I am available on skype if you need to chat or talk – Vipin Varghese Nov 26 '20 at 12:56
  • I found something interesting that when I tried this offload `ETH_RSS_IPV6_EX` with Intel I350 card, it gave the desired result by sending the scapy packet which is in the original post, similar to what is mentioned in [link](https://learn.microsoft.com/en-us/windows-hardware/drivers/network/rss-hashing-types#ndis_hash_ipv6_ex). So could this be that Mellanox is expecting something else or it is downright not working with it. ill try it on the ConnectX-6 card I have and see its behavior. – Ameer Usman Nov 27 '20 at 05:48
  • good, now you doing it right. Since you have a lot of NIC please experiement. – Vipin Varghese Nov 27 '20 at 05:59

2 Answers2

1

Okay, so guys I contacted Mellanox support and they have replied that they have checked with their DPDK expert and reported that they do not support IPv6 Extension headers support in RSS offloads, thus the reason zero hash was being calculated when IPV6_EX RSS offload was used.

The question I asked Mellanox was in regards to ConnectX-5 and ConnectX-6.

Thank you @Vipin for your support.

Ameer Usman
  • 95
  • 10
  • thank you for confirming my understanding that `ETH_RSS_IPV6_EX ` is just a generic place holder. It depends upon PMD and HW capability to support the same or not. – Vipin Varghese Dec 07 '20 at 14:40
0

As explained in the comments, ETH_RSS_IPV6_EX is a generic place holder. Not all drivers support the same. So depending upon PMD, driver and firmware extended features like RSS based on SRC-IP/DST-IP or some part of fields will be supported. example ETH_RSS_IPV6_TCP will be used if IPV6 has TCP in it and ETH_RSS_IPV6_TCP_EX will use if TCP is extended header.

note: igb and ixgbe supports _EX while i40e does not, you check intel NIC. DPDK mail thread discussion. Hence do not expect every NIC will has same features set for RSS

Vipin Varghese
  • 4,540
  • 2
  • 9
  • 25
  • @AmeerUsman, did you cross the links in the answers with NIC card under test. If you find this as right answer please accept and upvote. – Vipin Varghese Nov 30 '20 at 03:25
  • Actually I am still skeptical in regards to the `IPV6_EX` offload not being supported, as it is present in the mlx5_flow.h and also generates a proper output when performing & operation with its own legit RSS offload types. – Ameer Usman Nov 30 '20 at 06:02
  • not sure what do you mean @AmeerUsman, in your comments you mentioned `Intel i350` you got it working, which is the same as mentioned in Link shared. With respect to MXL5 you need to check with the vendor or vendor datasheet if the feature is supported. Then cross check whether DPDK PMD for the given version is supported. As requested by the question, the information about RSS IPV6_EX is shared. the question is not about MXL5 support or not – Vipin Varghese Nov 30 '20 at 06:21
  • I believe that is acceptable. Also I wanted to ask if you have ever come across any documentation describing the function of the offloads as mentioned in the 2nd point of the original post. – Ameer Usman Nov 30 '20 at 07:31