0

This is regarding Helloworld program in DPDK in custom linux running with VM player. I have installed DPDK19.11 in a Ubuntu VM and tried Hello world program. I have also allocated hugepages and mounted in /mnt/huge. It's working fine. The NIC driver is e1000.

Then I installed DPDK 19.11 using the same steps in a custom Linux distribution. It was not working well. I noticed that NIC driver info is mentioned as PCNET32 from AMD(somewhere in custom Linux creation this might be set).

Hence I have used rmmod command to remove PCNET32 and INSMOD command to load e1000.

When I put /sbin/lsmod | grep e1000 I am getting e1000 entry as well. But hello world program fails saying that

EAL: Detected 4 lcores
EAL: Detected 1 NUMA nodes
....
EAL: error allocating rte services array
EAL: FATAL rte_service_init() failed
EAL: rte_service_init() failed
PANIC in main
cannot init EAL

One more observation, after insmod command, when I put ethtool -i eth0 it says cannot get driver information: no such device. Earlier it used to display PCNET32. Now it should display e1000 in the same way as Ubuntu. If restart network service again, it shows PCNET32 again.

Can someone help me to understand these two things and guide me to resolve these issues?

Edit: This example is from DPDK helloworld. I have executed the program with the following command
./helloworld -l 0-3 -n 1 --no-pci --loglevel=eal,8
Log contents

EAL: Probing VFIO support  
EAL: IOMMU type 1 ( Type 1 ) is supported  
EAL: IOMMU type 7 ( sPAPR ) is not supported  
EAL: IOMMU type 8 (No-IOMMU ) is not supported  
EAL: VFIO support initialized  
EAL: Ask a virtual area of  bytes  
EAL: Virtual area found  
EAL: Setting up physically contiguous memory  
EAL: Setting Max no of open files to 4096  
EAL: Detected memory type:scoket_id :0  
EAL: Creating 4 segment lists  
EAL: Ask a virtual area of  bytes  
EAL: Virtual area found at 
EAL: Master lcore 0 is ready  
EAL: lcore 3 is ready  
EAL: lcore 1 is ready  
EAL: lcore 2 is ready  
EAL: NUMA is not supported  
EAL: alloc_seg():get_mempolicy: Function not implemented  
EAL: Ask a virtual area 
EAL: Virtual area found 
EAL: attempted to allocate 1 segments, but only 0 were allocated  
EAL: error allocating rte services array  
EAL: rte_service_init() failed  
PANIC in main() 
Cannot init EAL  
5:[./helloworld(_start_ _]  
4:[/lib/libc.so.6(_ _libc_start)  
3:[./helloworld() 
2:[./helloworld(__rte_panic)  
1:[./helloworld(rte_dump_stack)

I verified with ldd helloworld command. Libnuma is linked with helloworld.

I understand the issue may not be with NIC interface based on comments.
It seems insmod command is not enough to enable e1000 driver to NIC interface.

Just also want to understand how to enable Intel drivers such as e1000 with any of the interfaces by default or manually(using insmod) as it's required to leverage DPDK high performance.

I simply copied DPDK full installation from Ubuntu where I did build and did make helloworld with RTE_TARGET to custom Linux. It worked for MySQL, Java, Tomcat. Would that it be any issue?

Please let me know if you need more details.

Edit 2:
I have already created huge pages and mounted using the following steps after booting.

echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
mkdir /mnt/huge
mount -t hugetlbfs nodev /mnt/huge

Please find cat /proc/meminfo output

Hugepages Total = 1024
Hugepages Free = 1024
Hugepages Rsvd = 0
Hugepages Surp = 0
Hugepage Size 2048.

But helloworld program failed again

These are the same steps I carried out in Ubuntu as well. There it's working fine.

Then I tried as you suggested helloworld with --no-huge option.It works fine. Thanks for your help !
May I know is there any step missed in hugepage setup. How do I avail hugepage feature?

The other option could be building linux with different version of kernel, glibc etc in align with Ubuntu vm configuration to avail hugepages.

Would appreciate your suggestion.

Edit 3:

After adding ethernet0.virtualDev = "e1000" in vmx file for vm, intel e1000 NIC card is enabled by default.

vijay
  • 59
  • 2
  • 12
  • can you please confirm the error is found with DPDK example `helloworld`? Also please cleanup the question and share the log for `--no-pci --log-level=eal,8` – Vipin Varghese Sep 07 '20 at 09:44
  • based on the logs shared from your environment it looks like the Application is not able to find huge-pages. So please ensure you have created Huge pages (inside VM I am assuming you are setting up 2MB and not 1GB) and you have mounted the same. Use option `-m 256` to request for 256MB huge page. If it still have problem please add results for `cat /proc/cmdline` and dpdk application run with `--no-pci --no-huge` – Vipin Varghese Sep 08 '20 at 13:42
  • It looks like huge page memory is not getting identified, can you come over chat for love debug. @sunil would you like join too? – Vipin Varghese Sep 10 '20 at 22:47
  • @VipinVarghese. Thanks Vipin. Let me try with DPDK code and come back with more details. Anyway I am expected to work beyond hello world. This is a good chance to understand code. – vijay Sep 11 '20 at 05:14
  • Hi Vijay, is there an update on the issue? would like to debug this via chat? – Vipin Varghese Sep 18 '20 at 10:52
  • Hi Vipin, Thanks for your help. I thought of sorting out Nic card first since it's required for DPDK. You are right. After putting e1000 also, helloworld is having the same issue. Hence helloworld doesn't have anything with to do with NIC interface. Next I will work on DPDK and get some understanding before seeking your help. – vijay Sep 18 '20 at 14:23
  • Hi Vipin, Please find the update. I tried to find the issue based on your inputs to use log level=EAL,8. Then I also added some RTE_LOG messages. wIth --no huge option it works fine. In normal case, it's failing due to NUMA is not supported. – vijay Sep 25 '20 at 11:42
  • In normal case, it's failing due to NUMA is not supported. It goes into rte_service file then there is rte_calloc method is there. From there it's goes rte_malloc_socket then it's fails where code checks for numa_available condition in rte_memalloc.c in linux/eal. This could be some basic lack of understanding in custom linux setup as it works in Ubuntu vm. Any idea on this? – vijay Sep 25 '20 at 11:50
  • cat /sys/bus/pci/devices/0000\:xx\:00.x/numa_node this command gives value -1 in Ubuntu where as in Custom Linux numa_node file is not there. If required I can give more info or come in chat – vijay Sep 25 '20 at 11:53
  • can you come on chat or skype? – Vipin Varghese Sep 25 '20 at 13:56

2 Answers2

0

DPDK is generally compiled and tested in standard Linux distro's like Ubuntu/Fedro etc., so not guarantee it works as expected in your custom Linux machine. Also DPDK functions only on supported hardware in which I don't see AMD PCnet32 interface is available in the supported NICs unlike Intel e1000 NIC. Either virtual emulated or baremetal native NICs, should be supported by DPDK library is mandatory. Secondly, you cannot expect it to work when you perform NICs rmmod of original driver and modprobe any other driver.

Sunil Bojanapally
  • 12,528
  • 4
  • 33
  • 46
  • Thanks Sunil. Let me try to bring e1000 as part of custom Linux. It's based on LFS. – vijay Sep 07 '20 at 09:06
  • Hi Sunil, the test application used by Vijay is `example/helloworld`. This has nothing to do with NIC ports. The service_init failure mentioned, could be relate rte_service. Vijay should be running `--log-level=eal,8` for more details. – Vipin Varghese Sep 07 '20 at 09:20
  • I have been running DPDK on custom linux or Linux from scratch. One has to ensure the build dependency and libraries are met. Hence the answer shared is incomplete. Can you follow up with Vijay? – Vipin Varghese Sep 07 '20 at 09:27
  • @VipinVarghese, you are right, I missed the example being used. Will wait for an update from OP for right fix. – Sunil Bojanapally Sep 08 '20 at 12:10
0

based on the live debug with @vijay, the issue is not with DPDK libraries or DPDK build but with the environment in use. Current binaries for target Linux is not built via cross-toolchain. Instead Ubuntu is used to create binaries and copied to target.

Request to use DPDK cross build for libraries and PMD for the target. Suggested the flags be used as

export RTE_SDK=[dpdk parent]
export RTE_TARGET=[dpdk target]
export SYSROOT=[path to toolchain sysroot]
export ARCH=[target architecutre]
export RTE_KERNELDIR=[path to toolchain linux]
export CROSS=$CROSS_COMPILE
export EXTRA_CFLAGS="--sysroot=$SDKTARGETSYSROOT"
Vipin Varghese
  • 4,540
  • 2
  • 9
  • 25