Stack Overflow
Stack Overflow

Questions tagged [rdma]

RDMA refers to "Remote Direct Memory Access," which is a set of networking technologies typically used for high performance, low latency communication.

RDMA refers to "Remote Direct Memory Access," which is a set of networking technologies typically used for high performance, low latency communication.

RDMA networks have three main attributes:

  1. Asynchronous queueing: network operations are submitted by adding requests "work queues" which are asynchronously executed by hardware; when complete, the result is returned to software in a "completion queue."
  2. Kernel bypass: userspace processes can submit operations directly to the network adapter hardware without any system calls.
  3. One-sided operations: one system can read from or write into the memory of a remote system without involving any software on the remote system.
177 questions
2
votes
1 answer

RDMA program randomly hangs

Anyone out there who has done RDMA programming using the RDMA_CM library? I'm having a hard time finding even simple examples to study. There's an rdma_client & rdma_server example in librdmacm, but it doesn't run in a loop (rping does loop, but…
Yale Zhang
  • 1,447
  • 12
  • 30
2
votes
1 answer

python rdma "Hello World"

This program is my first attempt at Infiniband programming. I am using this library: https://github.com/jgunthorpe/python-rdma I am probably doing very funny things wrong. I am just trying to send a sequence of bytes "Hello, World" to a multicast…
Ivan
  • 7,448
  • 14
  • 69
  • 134
2
votes
1 answer

Benchmark tools for Soft-RoCE

I am new to SoftRoCE, I have installed OFED-1.5.2-rxe in my system. currently using kernel version 3.0.0+. Is there any specific benchmark tools for SoftRoCE. And is opensm is mandatory for SoftRocE? while i am trying to run rdma_bw vvdn@vvdn:~$…
Dinesh Kb
  • 31
  • 2
2
votes
2 answers

How to use SRQ with different connections in libibverbs

How to use SRQ when connected to more than one connections. lets say, there are three connections namely process 0, 1 and 2. for creating SRQ, we need to call struct ibv_srq *ibv_create_srq(struct ibv_pd *pd, struct ibv_srq_init_attr…
Aunn Raza
  • 468
  • 2
  • 12
2
votes
1 answer

How does GPUDirect enforce isolation on a shared device

I have been reading here https://developer.nvidia.com/gpudirect about GPUDirect, In there example there is a network card attached to the PCIe together with two GPU's and a CPU. How is isolation enforced between all clients trying to access the…
yonigo
  • 987
  • 1
  • 15
  • 30
2
votes
2 answers

How to write a Hello World iWarp application?

I am trying to write a small program demonstrating Remote DMA over iWarp. I have the softiwarp Linux kernel module loaded and the userspace library compiled. I am looking for documentation or sample code that explains setting up a connection and,…
Julian Stecklina
  • 1,271
  • 1
  • 10
  • 24
1
vote
0 answers

nvidia_p2p_get_pages() failing with error code -22

I am implementing NVIDIA GDS with the following hardware config: Ubuntu 22.04 CUDA 12.1 Nvidia Drivers 530.30.2 MLNX driver - 5.8.0 NVIDA GeForce RTX 3090 Samsung 980 DC NVMe drive. IOMMU is disabled PCIe bar has been resized to that of VRAM…
utkrisht patesaria
  • 11
  • 2
1
vote
1 answer

How to dynamically modify QP rate limit in ConnectX-6 Infiniband/VPI adapter

I have a ConnectX-6 Infiniband/VPI Adapter. I can setup the hardware rate limit when creating a qp like this: ... ibv_qp_attr.ah_attr.static_rate = 7; // set qp rate limit to 40Gbps ... ibv_modify_qp(qp, &ibv_qp_attr, flags); ... But I cannot…
Dillion Wang
  • 362
  • 3
  • 18
1
vote
1 answer

RDMA read and write data placement/visibility semantics

I am trying to get more details on the RDMA read and write semantics (especially data placement semantics) and I would like to confirm my understanding with the experts here. RDMA read : Would the data be available/seen in the local buffer, once…
user718134
  • 13
  • 2
1
vote
1 answer

Infiniband RDMA completion queues with multithreading

I'm learning how to use RDMA via Inifniband and one problem I'm having is using a connection with more than 1 thread because I cant figure out how to create another completion queue so the work completions get mixed up between the threads and it…
CommanderLake
  • 51
  • 9
1
vote
0 answers

FPGA to DMA to RDMA

I am trying to send data generated from my FPGA card out to an IB device. I want the latency to be as low as possible, so I am thinking this may be the data path. FPGA --> DMA via scatter/gather DMA into Memory Buffer --> RDMA into a ConnectX-6 card…
bchang32
  • 27
  • 3
1
vote
1 answer

How does the remote Peer handles data send by a RDMA Write operation

I have difficulties to understand how and in which cases RDMA operations are used. Let's say we have a server and a client. The client writes data via rdma-write to the memory region of the server. Since the server doesn't get any notification that…
user11416826
1
vote
1 answer

RDMA memory semantics for READ/WRITE operations and local memory operations

I am seeing unexpected results from RDMA reads that make me doubt my understanding of the RDMA read and write semantics. I'm trying to implement message passing in RMDA in a manner similar to L5, but am running into issues that look like memory…
razeh
  • 2,725
  • 1
  • 20
  • 27
1
vote
1 answer

What is the logic behind setting valid region half the size of PSN range?

In order to make it possible for the responder to distinguish duplicate packets from out of order packets, a given send queue shall have a series of PSNs no greater than 8,388,608 outstanding at any given time. Therefore, a send queue shall have no…
Barney_su
  • 43
  • 6
1
vote
1 answer

What's the difference between Doorbell and Doorbell Record?

I'm new to RDMA technology and I can't figure out what's the difference between doorbell and doorbell record. In my opinion, when Software posts its work request, it rings the doorbell register to notify the hardware to fetch the work request. I…
MitsuiYang
  • 21
  • 2
1 2 3
…
11 12