Questions tagged [rdma]

RDMA refers to "Remote Direct Memory Access," which is a set of networking technologies typically used for high performance, low latency communication.

RDMA refers to "Remote Direct Memory Access," which is a set of networking technologies typically used for high performance, low latency communication.

RDMA networks have three main attributes:

  1. Asynchronous queueing: network operations are submitted by adding requests "work queues" which are asynchronously executed by hardware; when complete, the result is returned to software in a "completion queue."
  2. Kernel bypass: userspace processes can submit operations directly to the network adapter hardware without any system calls.
  3. One-sided operations: one system can read from or write into the memory of a remote system without involving any software on the remote system.
177 questions
0
votes
0 answers

ibverbs: wc.status = IBV_WC_LOC_ACCESS_ERR (8)

I have a simple program which transfers data from a client to a server using rdma. I have 5 qps and the length of send queue and receive queue are both 50. Under this way, my program runs well, it can thansfers 1million requests. When i set th…
0
votes
0 answers

SMB Direct Between WIN10 Enterprise and WIN 10 Pro

I want to configure SMB Direct using IWARP on my Computer. I use Network Adapter T6225-CR both side, and i can find this Adapter on my device Manager. And Server OS is Window 10 enterprise and Client Window 10 Pro. When i type command…
holee
  • 1
  • 1
0
votes
0 answers

How to fix a python "pyverbs" application with RDMA write getting a not acknowledge (NACK)

Im currently working on a RoCE (RDMA over Converged Ethernet) python application with the library pyverbs. First, i want to do a simple loopback test with an RDMA Write. I tested the setup with ib_write_bw from perftest, which worked like a…
RealGhost
  • 1
  • 2
0
votes
0 answers

GPUDirect RDMA out of range pin address by Quadro p620

I want to implement FPGA-GPU RDMA by nvidia quadro p620. Also, I used common PCIe BAR resources(BAR0 - BAR1 - BAR2) for FPGA registers and other chunk controllers handling which is independent from RDMA in my custom driver. PCIe managements are OK…
SkyCyborg
  • 1
  • 2
0
votes
0 answers

Is RDMA/DMA performance impacted by 'Region 0' size

I am using the Mellanox ConnectX-6 NIC and the configuration is as shown below: (lspci -vv) a1:00.0 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6] Subsystem: Mellanox Technologies Device 0028 Control: I/O- Mem+…
Vaishakh
  • 67
  • 5
0
votes
0 answers

How to use RDMA between two RDMA capable instances on Azure

I have created 2 Standard HB120rs v2(RDMA capable) Virtual Machines on Azure with centos-based HPC, On "ifconfig" I am getting below ips for 2 machines I tried testing rping using the below commands On instance 1 executed rping -v -s -a…
JibinNajeeb
  • 784
  • 1
  • 10
  • 31
0
votes
0 answers

Running TCP over RDMA /IBand with MPI

community! For instance, we have 2 nodes with MPI interconnection and the next set of interfaces: ib0 (InfiniBand), eth10 (Ethernet) and lo. To run MPI on mlx4 device with RDMA we use the next command: mpirun --allow-run-as-root --host host1,host2…
0
votes
0 answers

errno 14 when trying to implement buffered MPI_Get

I tried to implement a buffered version of MPI_Get, so no global fence is necessary when one process gets something from only one other process. However, I get this error: Read -1, expected 40, errno = 14 I used openmpi version 4.1.2, and ran the…
0
votes
1 answer

Rdma infiniband cannot open hosts (iberror: discovery failed) Port state: Down

I am facing an issue while configuring rdma and Infiniband on my two nodes. Both of these two nodes are connected and I have installed the recommended software libraries and packages required. But my port status is down and physical state is…
DumbLoawai
  • 11
  • 4
0
votes
1 answer

How can I use the Memory inside NICs?

As explained in this paper ASPLOS'22 The benefits of general-purpose on-NIC memory
lizebin
  • 3
  • 2
0
votes
1 answer

Getting data along with an ACK

I am using InfiniBand/RDMA for this. My client machine is sending a block to the server (via RDMA Send), and I want the client to get back a special 8 byte acknowledgement. I know the server could send it along separately, but that would duplicate…
Mihir Shah
  • 39
  • 1
  • 5
0
votes
1 answer

Can a RDMA write verb cause interrupt on the targeted server?

Hi i am wondering whether a RDMA write verb can cause interrupt on the targeted server? i don't think this is how people would use it but i am just curious whether the RDMA can or can't
zbhhbz
  • 19
  • 2
0
votes
1 answer

RDMA write and RDMA write with immediate

I am trying to get more details on the RDMA write semantics. I have the following questions on the same. How the RDMA write and RDMA write with immediate works? If I use the RDMA write operation, can I read data from the local buffer on the peer…
0
votes
1 answer

How to get symbol info from libmlx5-rdmav2.so?

I am using Vtune to analyze my program. And the Vtune reports that Function Module CPU Time % of CPU Time(%) ---------------------------- ----------------- --------- ---------------- func@0x23dc0 …
0
votes
0 answers

Deploy a ceph cluster with RDMA using docker

I'm trying to deploy a ceph cluster on Ubuntu16.04, using images ceph/daemon:latest-luminous. It worked out perfectly at first without using RDMA, and then after I change the configuration and I entered the container inside and input ceph -s, it…