0

I would need some recommendation about the benefits of using a dedicated Intel Ethernet card versus simply using the integrated Ethernet of the motherboard. The typical use case would be the following:

  1. Payload of around 100 KB per message,
  2. Frequency of around 1000 messages per second maximum,
  3. Ideally, the handling of these messages should consume CPU time as minimum as possible (the CPU will be used for real time computation in a parallel process),
  4. Linux based server.

For the integrated Ethernet solution, I could have access to a mainstream motherboard (not server grade) equipped with an (integrated) Intel I225V or a Realtek RTL8125BG.

For the solution with an external NIC, I was thinking about the Intel X550-T1.

So, which solution would be better ?

In the case where a dedicated card is better for my use case, what PCI-express port should be favored ? Should the port used be as close as possible to the CPU ?

Henry
  • 15
  • 4

2 Answers2

1

There isn't a difference in latency between onboard and dedicated NIC. Both of these use the PCI-express. Likely the dedicated NIC will be connected to internal PCI-e controller in CPU. Onboard integrated NIC can be connected to the chipset integrated conntroller, not directly to the CPU. But the difference in latencies of transmit data from NIC to memory is almost absent.

If you really care about performance and latency of processing, check next points:

  • NICs uses the DMA to transmit data to the main memory, so use the memory with highest frequency
  • Newer version of PCI-e is better (more speed - less time for transmit data)
  • CPU freq is more important than number of cores.
  • In NUMA systems the data should be processed at a local node.
  • Don't forget disable the power-saving features
  • Use the software with the kernel-bypass like Intel DPDK
  • Read about Smart-NIC or FPGA-based DPU to offload the processing
Anton Danilov
  • 5,082
  • 2
  • 13
  • 23
1

In order to compare onboard vs PCIe NIC, one has to use the same ASIC model with the same firmware to eliminate external factors. With this assumption let us take into account for X550-T1 ASIC. Some of the factors which affects onboard vs PCIe card are

  1. Number of dedicated PCIe lanes.
  2. NUMA CPU threads accessing the NIC.
  3. Firmware available for onboard NIC (certain motherboard vendors will ship custom firmware only).
  4. read and write buffer size from the PCI enumeration.
  5. low latency mode (example -a 0000:88:00.0,rx_low_latency=1).

Note: all other factors like memory, CPU, disk, I/O, UNCORE are all system or CPU related and does not affect onboard vs external PCIe NIC.

Vipin Varghese
  • 178
  • 5
  • Thanks for the insight. From what you said, considering there are many unknowns about the implementation of the onboard network chip, both software and hardware, would it be reasonable to consider that, on average, it is better to use an external card, like the Intel one cited above, where the driver and firmware are known, and where the connection can be chosen (in the sense a fast PCIe port) ? – Henry Jan 10 '22 at 20:13
  • @Henry, there is the motherboard that supports X710 and X550 as onboard NIC ASIC also. Hence comparison is to be done against similar family external NIC, since all other factors remain the same. A good example is Intel Xeon-D where integrated 10Gbps and 25Gbps are present. Based on the use case it always better to explore as price and PCIe slots can be saved with integrated NIC with SoC. – Vipin Varghese Jan 11 '22 at 03:53
  • @Henry if you find the answer useful please accept and upvote. – Vipin Varghese Jan 20 '22 at 17:05
  • Sorry, I had already accepted the first answer and unfortunately I do not have enough reputation to upvote your answer. – Henry Jan 23 '22 at 22:33
  • @Henry thanks for the update. I hope you are clear in the differences between onboard vs external for the same ASIC query? – Vipin Varghese Jan 24 '22 at 02:08