DPDK: HW offloaded calculation of UDP checksum not working

Question

I am working with DPDK version 18.11.8 stable on Linux with an Intel X722 NIC.

My app works fine if I calculate IP and UDP checksums in software but I get a segmentation fault if I calculate in hardware. Here is my code:

local_port_conf.txmode.offloads  = local_port_conf.txmode.offloads | DEV_TX_OFFLOAD_IPV4_CKSUM  | DEV_TX_OFFLOAD_UDP_CKSUM;

mb->ol_flags |= PKT_TX_IPV4 | PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM; 
mb->l2_len = sizeof(struct ether_hdr);
mb->l3_len = sizeof(struct ipv4_hdr);
mb->l4_len = sizeof(struct udp_hdr);        
p_ip_hdr->hdr_checksum = 0;
p_udp_hdr->dgram_cksum = rte_ipv4_phdr_cksum((const ipv4_hdr*)(mb->l3_len), mb->ol_flags);

The rte_ipv4_phdr_cksum() call is mysterious, have I understood what to do correctly?

Understandably, the C++ compiler gaves a warning:

warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
         p_udp_hdr->dgram_cksum = rte_ipv4_phdr_cksum((const ipv4_hdr*)(ptMbuf->l3_len), ptMbuf->ol_flags);
                                                                                      ^

What is wrong with my code?

Answer 1

Following are the steps required for HW checksum offload for IP and UDP in DPDK:

1. Check whether hardware supports HW checksum offload.
2. Prepare the packet IP and UDP header fields.
3. Initialize the checksum fields. In case of partial UDP checksum offload support, initialize the UDP checksum with the checksum of the IP pseudo-header.

Note: The Intel NIC X553, X710 & X722 (and probably others), only support partial UDP checksum offload,

[EDIT based on the query in the comment from @maxschlepzig] Are there NICs which support full UDP checksum offloading? If yes, is there some DPDK way to check for full support?

[Answer] Yes, there is an easy way to check. In NIC features TABLE 1.1 highlights if checksum offload is available, partial or absent. But the real catch is to check release notes for NIC specific firmware as the HW ASIC or fixed functions are enabled or disabled for the same.

which requires a snippet like this:

/* during port configuration */
txmode = {
    .offloads = DEV_TX_OFFLOAD_IPV4_CKSUM | DEV_TX_OFFLOAD_UDP_CKSUM
};

/* during device configuration */
if (!(dev_info.tx_offload_capa & DEV_TX_OFFLOAD_UDP_CKSUM) || !(dev_info.tx_offload_capa & DEV_TX_OFFLOAD_IPV4_CKSUM)) {
    rte_panic(" offload not supported");
}

/* mb is mbuf packet to transmit */
mb->ol_flags = PKT_TX_IPV4 | PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM;
mb->l2_len = sizeof(struct ether_hdr);
mb->l3_len = sizeof(struct ipv4_hdr);      

struct rte_ether_hdr *eth_hdr = rte_pktmbuf_mtod(mb, struct rte_ether_hdr *);
struct ipv4_hdr *p_ipv4_hdr = (struct ipv4_hdr*) ((char *)eth_hdr + sizeof(struct ether_hdr));
struct udp_hdr *p_udp_hdr = (struct udp_hdr *)((unsigned char *) + sizeof(struct ipv4_hdr));

/* update v4 header fields with version, length, ttl, ip and others */
/* update udp headers */

/* in case hardware offload is unavailable */ 
p_udp_hdr->dgram_cksum = rte_ipv4_udptcp_cksum(p_ip_hdr, p_udp_hdr);
p_ip_hdr->hdr_checksum = rte_ipv4_cksum(p_ip_hdr)

/* otherwise, equivalent offloaded checksum computation */ 
p_udp_hdr->dgram_cksum = rte_ipv4_phdr_cksum(p_ipv4_hdr, ol_flags);
p_ip_hdr->hdr_checksum = 0; 

[EDIT-2] thanks to @maxschlepzig in pointing out l4_len is not required for UDP Checksum offload. This is the right information l4_len is only required for TCP Segment offload for TX.

Based on the live debug with @DavidA, the segfault reason were identified to incorrect usage and in the comments, the code is updated as

@VipinVarghese Thanks so I now have: struct ipv4_hdr* p_ipv4_hdr = rte_pktmbuf_mtod(mb, struct ipv4_hdr *) + mb->l2_len; p_udp_hdr->dgram_cksum = rte_ipv4_phdr_cksum(p_ipv4_hdr, mb->ol_flags); but the udp checksum is still incorrect.

Hi David, I believe you have not st the right values. Will update in couple of hours – Vipin Varghese

one of the action items that was shared with David was updating the pseudo code as the real question revolved around DPDk: HW offloaded calculation of UDP checksum not working. Thanks, @maxschlepzig for explaining it all, as comments and live debug can be easily missed out by many.

Answer 2

The function rte_ipv4_phdr_cksum() is documented as:

/**
 * Process the pseudo-header checksum of an IPv4 header.
 *
 * The checksum field must be set to 0 by the caller.
 *
 * Depending on the ol_flags, the pseudo-header checksum expected by the
 * drivers is not the same. For instance, when TSO is enabled, the IP
 * payload length must not be included in the packet.
 *
 * When ol_flags is 0, it computes the standard pseudo-header checksum.
 *
 * @param ipv4_hdr
 *   The pointer to the contiguous IPv4 header.
 * @param ol_flags
 *   The ol_flags of the associated mbuf.
 * @return
 *   The non-complemented checksum to set in the L4 header.
 */
static inline uint16_t
rte_ipv4_phdr_cksum(const struct rte_ipv4_hdr *ipv4_hdr, uint64_t ol_flags)

So when you cast like this

p_udp_hdr->dgram_cksum = rte_ipv4_phdr_cksum((const ipv4_hdr*)(mb->l3_len),
        mb->ol_flags);

the ipv4_hdr parameter contains an nonsensical address. Thus, this can't work. And this is the root cause for your segmentation fault (-> an invalid address is dereferenced).

For computing the checksum of the pseudo-header you have to supply your original IP header, thus, just call it like this:

p_ip_hdr->hdr_checksum = 0;
p_udp_hdr->dgram_cksum = rte_ipv4_phdr_cksum(p_ip_hdr, mb->ol_flags);

The rte_ipv4_phdr_cksum() then internally builds a temporary pseudo header (based on the supplied p_ip_hdr) in order to compute its checksum.

NB: You don't have to assign to mb->l4_len for UDP checksum offloading. Setting l2_len and l3_len is sufficient.