Questions tagged [rdma]

RDMA refers to "Remote Direct Memory Access," which is a set of networking technologies typically used for high performance, low latency communication.

RDMA refers to "Remote Direct Memory Access," which is a set of networking technologies typically used for high performance, low latency communication.

RDMA networks have three main attributes:

  1. Asynchronous queueing: network operations are submitted by adding requests "work queues" which are asynchronously executed by hardware; when complete, the result is returned to software in a "completion queue."
  2. Kernel bypass: userspace processes can submit operations directly to the network adapter hardware without any system calls.
  3. One-sided operations: one system can read from or write into the memory of a remote system without involving any software on the remote system.
177 questions
4
votes
4 answers

Infiniband in Java

As you all know, OFED's Socket Direct protocol is deprecated and OFED's 3.x releases do not come with SDP at all. Hence, Java's SDP also fails to work. I was wondering what is the proper method to program infiniband in Java? Is there any portable…
RoboAlex
  • 4,895
  • 6
  • 31
  • 37
4
votes
2 answers

In Infiniband, what mapping in PCIe-BAR, the internal buffer of Infiniband card or the remote computer's RAM?

As we know, Infiniband allows RDMA - direct access to the memory of the remote computer. It is also known, that the PCI-Express (endpoint) devices, including the PCIe-card Infiniband, are able to communicate in two ways: IO Ports (in / out)…
Alex
  • 12,578
  • 15
  • 99
  • 195
4
votes
3 answers

What is the maximum length of the cable can be for infiniband(RDMA)?

What is the maximum length of the cable can be for infiniband(RDMA)? For example. Here have said, that it can be up to 10 M for a four-channel copper cable. And using the Connects Cables using fiber, which fully electrically compatible with standard…
Alex
  • 12,578
  • 15
  • 99
  • 195
3
votes
1 answer

What is the replacement of libibverbs/librdmacm for Windows?

We have our application running on Linux and using RMDA (Infiniband) interface for communication between two modules. Now we like to support our application on Windows and hence looking for IB Verbs replacement. We tried installing Mellanox Drivers…
dudhaniss
  • 51
  • 5
3
votes
1 answer

RDMA between GPU and remote host

Is it possible to perform an RDMA operation between a GPU and a remote host? The documentation online on Nvidia's website only talks about doing RDMA between GPUs and does not mention the possibility of doing this between GPU-host. Note: I have…
JC1
  • 657
  • 6
  • 21
3
votes
1 answer

How to: Azure OpenMPI with Infiniband - Linux

I am new to using Microsoft Azure for scientific computing purposes and have encountered a few issues whilst setting up. I have a jump box set-up that acts as a license server for the software that I whish to use, is also has a common drive to store…
Owain1602
  • 31
  • 2
3
votes
1 answer

What's the doorbell in InfiniBand?

I am reading a document ablut InfiniBand, which is called 《InfiniBand™ Host Channel Adapter Verb Implementer’s Guide》--Intel, in Chapter 6.2.2, it mentions a concept called "Doorbell", I haven't found any information about this concept in the IB…
3
votes
0 answers

What is the difference between SDP and VMA?

As known for Infiniband or RoCE-Ethernet we can use: VMA - Voltaire Messaging Accelerator (VMA) Mellanox's Messaging Accelerator (VMA) boosts performance for message-based and streaming applications... The result is a reduction in latency by as…
Alex
  • 12,578
  • 15
  • 99
  • 195
3
votes
1 answer

Difference between Infiniband and RDMA

Man pages of linux command ibstat and ibv_devinfo say ibstat - query basic status of InfiniBand device(s) ibv_devinfo - query RDMA devices In this context, what is the difference between Infiniband devices and RDMA devices? And in what…
Sagar Jha
  • 1,068
  • 4
  • 14
  • 24
3
votes
2 answers

Java Sockets on RDMA (JSOR) vs jVerbs performance in Infiniband

I have basic understanding of both JSOR and jVerbs. Both handle limitations of JNI and use fast path to reduce latency. Both of them use user Verbs RDMA interface for avoiding context switch and providing fast path access. Both also have options for…
phoenix
  • 3,069
  • 3
  • 22
  • 29
3
votes
1 answer

posted recv buffers returned synchronously or async after calling rdma_disconnect()

When calling rdma_disconnect(), do I get completion queue events for all posted recv work requests before rdma_disconnect() returns, or should I expect them to come in after rdma_disconnect() has returned?
Andy Grover
  • 608
  • 3
  • 6
3
votes
1 answer

Can I use in RDMA via Infiniband Load/Store access from GPU2-Cores to GPU1-RAM in the different PCIe-Bus?

As said here: https://developer.nvidia.com/gpudirect I can access from GPU0-Core to the GPU1-RAM on the Same PCIe Bus: Load/Store cudaMemcpy() It named as "NVIDIA GPUDirect v2 Peer-to-Peer (P2P) Communication Between GPUs on the Same PCIe Bus…
Alex
  • 12,578
  • 15
  • 99
  • 195
3
votes
1 answer

Are there any mechanisms available to use LDMA or RDMA on Windows CLR based platform?

I want to be able to make use of LDMA or RDMA on a Windows CLR-based application (.NET). I have low-latency applications being developed with C++ and those applications will receive their data using LDMA or RDMA. I'd like our CLR applications to…
Ajaxx
  • 2,500
  • 4
  • 27
  • 38
3
votes
2 answers

efficient one-sided communication

I am looking for an efficient one-sided communication library that can be used to implement RDMA efficiently. Currently, I've looked at MPI-2 implementations such as MPICH2 and also MVAPICH2 (InfiniBand Architecture based implementation of MPI-2…
LeTex
  • 1,452
  • 1
  • 14
  • 28
2
votes
0 answers

How irdma works?

After upgrading to Rocky Linux 8.6 I noticed that it was consuming more memory. After some time I figured out why this is happening. The driver of the installed Intel E810 (ice) network card is causing the irdma module in both of my servers: lsmod…
akashavkin
  • 331
  • 1
  • 3
  • 6
1
2
3
11 12