1

I have a ConnectX-6 Infiniband/VPI Adapter. I can setup the hardware rate limit when creating a qp like this:

...
ibv_qp_attr.ah_attr.static_rate = 7; // set qp rate limit to 40Gbps
...
ibv_modify_qp(qp, &ibv_qp_attr, flags); 
...

But I cannot dynamically change the qp rate limit later using the above code after creating the qp.

I also checked the ibv_modify_qp_rate_limit() API, but it keeps return EINVAL when I try to set the rate limit using this API:

struct ibv_qp_rate_limit_attr rl_attr;
memset(&rl_attr, 0, sizeof(rl_attr));
rl_attr.rate_limit = 100;
ibv_modify_qp_rate_limit(qp, &rl_attr); // returns EINVAL

Am I using the API right? How can I dynamically change the hardware rate limit of a qp? (or set the global hardware rate limit).

Dillion Wang
  • 362
  • 3
  • 18

1 Answers1

0

The rate_limit attribute is in Kbps. I suspect 100 Kbps is too low a limit for the device to handle (the original static_rate interface can only go down to 1X single data rate, which is 2.5 Gbps). You can check by using the

ibv_query_device_ex(context, &device_attr)

API and making sure your requested rate limit is between

device_attr.packet_pacing_caps.qp_rate_limit_min

and

device_attr.packet_pacing_caps.qp_rate_limit_max
Roland
  • 6,227
  • 23
  • 29
  • qp_rate_limit_min and qp_rate_limit_max are both 0 in my environment. Does this mean this device does not support limiting rate? – Dillion Wang Jun 23 '22 at 01:11
  • I'm not sure. From looking a bit more at the ibv_query_device_ex() documentation, it seems that API may be for raw packet QPs only. I'm not sure there's any way to modify the static rate for a connected QP after the QP is set up, although it may be possible to use the alternate path mechanism to set up a second destination with the same address but a different static_rate and then flip over to that. – Roland Jun 24 '22 at 01:23