Stack Overflow
Stack Overflow

Questions tagged [infiniband]

InfiniBand is a high-speed switched fabric communications link technology used in high-performance computing and enterprise data centers.

InfiniBand is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include scalability, high throughput, low latency, quality of service and failover. The InfiniBand architecture specification defines a connection between processor nodes and high performance I/O nodes such as storage devices. Infiniband host bus adapters and network switches are commonly manufactured by both Mellanox and Intel.

178 questions
3
votes
1 answer

Difference between Infiniband and RDMA

Man pages of linux command ibstat and ibv_devinfo say ibstat - query basic status of InfiniBand device(s) ibv_devinfo - query RDMA devices In this context, what is the difference between Infiniband devices and RDMA devices? And in what…
Sagar Jha
  • 1,068
  • 4
  • 14
  • 24
3
votes
1 answer

What is Tag-matching interface?

I heard that PSM is a library supporting tag-matching. What is Tag-matching interface? Why is tag-matching important for performance in the context of MPI?
kiriloff
  • 25,609
  • 37
  • 148
  • 229
3
votes
2 answers

Java Sockets on RDMA (JSOR) vs jVerbs performance in Infiniband

I have basic understanding of both JSOR and jVerbs. Both handle limitations of JNI and use fast path to reduce latency. Both of them use user Verbs RDMA interface for avoiding context switch and providing fast path access. Both also have options for…
phoenix
  • 3,069
  • 3
  • 22
  • 29
3
votes
2 answers

InfiniBand network performance

I am measuring the performance of InfiniBand using iperf. It's a one-to-one connection between a server and a client. I measured the bandwidth changing number of threads which request Network I/Os. ( The cluster server has: "Mellanox ConnectX-3 FDR…
syko
  • 3,477
  • 5
  • 28
  • 51
3
votes
3 answers

Can anybody please explain to me the relationship between libibverbs and librxe?

I am struggling to understand the relationship between libibverbs and librxe and the low-level kernel driver for the HCA. Specifically, I have the following doubts : When a packet arrives on the HCA, the low-level kernel driver passes the packet…
byslexia
  • 329
  • 2
  • 8
3
votes
1 answer

Can I use in RDMA via Infiniband Load/Store access from GPU2-Cores to GPU1-RAM in the different PCIe-Bus?

As said here: https://developer.nvidia.com/gpudirect I can access from GPU0-Core to the GPU1-RAM on the Same PCIe Bus: Load/Store cudaMemcpy() It named as "NVIDIA GPUDirect v2 Peer-to-Peer (P2P) Communication Between GPUs on the Same PCIe Bus…
Alex
  • 12,578
  • 15
  • 99
  • 195
3
votes
0 answers

Using ZeroMQ on Infiniband SDP

I am trying to run the ZeroMQ's local_thr/remote_thr on SDP (infiniband) compiled on MSVS2012. But it's not connecting. On IPoIB it is working properly. Operating System is Windows Server 2008 R2. On further investigation I found out that select()…
Eiraj
  • 31
  • 3
3
votes
1 answer

How to get Infiniband RX/Tx info

Is there any tools which gather and show infiniband RX/TX info ?
D3Hunter
  • 1,329
  • 10
  • 21
2
votes
1 answer

ipoib: Check if an infiniband node with an given iboip is reachable

I need to check if for an given ipoib IP address the associated IB node is connected to the infiniband switch. The idea is to get the guid of the IB port to which the IP packets are send and check with a command like ibnodes if the port is…
fho
  • 470
  • 5
  • 12
2
votes
1 answer

Multithreaded use of a single QP vs multiple QPs to improve throughput

I am using RDMA writes in my application and want to improve throughput. Currently, I have a single thread using my queue pair. I was wondering, what is a more standard way (or what are the advantages of each): Creating more connections with the…
Mihir Shah
  • 39
  • 1
  • 5
2
votes
1 answer

RDMA Read protection for local memory operations

I have the following scenario: My server allocates a buffer of 1MB, which is periodically updated and written to (about every 50ms). The client is connected to the the server via Infiniband and periodically reads that buffer via RDMA Read…
hey0
  • 43
  • 6
2
votes
1 answer

What is the difference b/w RDMA_CREATE_QP/RDMA_CREATE_EP and IBV_CREATE_QP?

As far as I understand, IBV_CREATE_QP() and RDMA_CREATE_QP literally do the same thing. If I have called ibv_create_qp() first, do I still need to call the other one when I'm establishing a connection using ibrdmacm API? Moreover, diff b/w create…
Barney_su
  • 43
  • 6
2
votes
1 answer

RDMA over native Infiniband

Is it possible to use RDMA over native InfiniBand without IPoIB (using just guid or lid)? I have checked Infiniband addressing - host names to IB address without IBoIP and in RDMA Connection Manager source files to create connection between ports…
mat300
  • 23
  • 3
2
votes
1 answer

Multihomed Spark Cluster

I am working on setting up a Spark cluster in a multihomed network situation and have run into some problems. I'll start with the physical configuration. I have 12 nodes all in a rack that have an inter-rack 100G infiniband network using ipoib and a…
Pumices
  • 315
  • 1
  • 10
2
votes
0 answers

rdma unexpected CM event 3 - RDMA_CM_EVENT_ROUTE_ERROR

I am hosting an Infiniband server in a linux machine and also I have created a client and connected to that service in the same machine.This works fine most of the time. But in one instance when I was trying to connect to that server from the same…
Kethiri Sundar
  • 480
  • 2
  • 12
1 2
3
…
11 12