0

When I am writing message length is more than 1024B(mtu), it failed in softroce mode, pls help check why.

Using the standard tool ib_write_lat to test:

when ib_write_lat -s 1024 -n 5

When ib_write_lat -s 1025 -n 5, it fails.

My softroce version in in Red Hat Enterprise Linux Server release 7.4 (Maipo)

Is it a bug in softroce?

krlzlx
  • 5,752
  • 14
  • 47
  • 55
liaotianyu
  • 1
  • 1

1 Answers1

0

No it isn't a bug. I had similar problems.

  • What did you configure at your interface configuration?
  • I expect that you have a MTU of 1500 Bytes configured (or leaved the default value), this will result in RoCE using 1024. If you configure your interface MTU to 4200 you can use the ib_write_lat command with up to 4096 bytes.

InfiniBand protocol Maximum Transmission Unit (MTU) defines several fix size MTU: 256, 512, 1024, 2048 or 4096 bytes.

RoCE based application that uses RDMA that runs over Ethernet should take into account that the RoCE MTU is smaller than the Ethernet MTU. (normally 1500 is the default).

https://community.mellanox.com/docs/DOC-1447

Florian Berndl
  • 1,136
  • 8
  • 27