How to build perftest and run latency tests on RDMA

Question

I'm trying to build the perftest C library to run some latency tests over the RDMA protocol.

My steps:

• I downloaded the library from their GitHub and unzipped it on the box on which I want to run the server for the tests

• I executed:

  ./autogen

  ./configure

  make clean && make V=1

• I tried to run a test

  ./ib_send_lat --duration=30 -H

..but I get the following error:

Port number 1 state is Down
Couldn't set the link layer
Couldn't get context for the device

What am I doing wrong?

Running other tests that use the RDMA protocol (eg. the ones in Accelio) the system shows no problems.

Also, I runned a few diagnostic tools and everything seems configured correctly already:

• Verified that RDMA kernel part is loaded (ibuverbs present)

• Verified that RDMA devices are available

• Verified that RDMA devices can be accessed (the second one says PORT_ACTIVE)

• Verified that traffic over RDMA is working sending messages from a client to a server using rping

Any suggestions? (I opened an issue on their GitHub but I got no reply yet)

Thanks

Answer 1

I suspect the problem is the perftest tools are defaulting to using the device that's in the PORT_DOWN state.

Try specifying the specific device to use with the '-d' option. eg:

ib_send_lat -d mlx5_0 ...

(This will probably need to be done on both sides of the link.)