Stack Overflow
Stack Overflow

Questions tagged [infiniband]

InfiniBand is a high-speed switched fabric communications link technology used in high-performance computing and enterprise data centers.

InfiniBand is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include scalability, high throughput, low latency, quality of service and failover. The InfiniBand architecture specification defines a connection between processor nodes and high performance I/O nodes such as storage devices. Infiniband host bus adapters and network switches are commonly manufactured by both Mellanox and Intel.

178 questions
0
votes
1 answer

RDMA Fast Memory Registration (FMR)

I'm developing a system that uses RDMA extensively (on Mellanox hardware) and would like to be able to register memory regions more efficiently/faster. I have taken a look into Fast Memory Registration and I have a few questions: Is FMR going away?…
JC1
  • 657
  • 6
  • 21
0
votes
1 answer

Issue in SA query Infiniband

What is the parameter context passed to the function ib_sa_path_rec_get from ib_sa?
S. Salman
  • 590
  • 1
  • 6
  • 22
0
votes
1 answer

CUDA-aware MPI for two GPUs within one K80

I am trying to optimize the performance of a MPI+CUDA benchmark called LAMMPS (https://github.com/lammps/lammps). Right now I am running with two MPI processes and two GPUs. My system has two sockets and each socket connects to 2 K80. Since each K80…
silence_lamb
  • 377
  • 1
  • 3
  • 12
0
votes
1 answer

Issue with ibv_poll_cq() & ib_poll_cq()

Client Side: ib_poll_cq(cq,1,&wc){ if(wc.status == IB_WC_SUCCESS) printk("Successful\n"); else printk("Failure: %d\n", wc.status); } Server Side: do { num_comp = ibv_poll_cq(s_ctx.recv_cq, 1, &wc); } while (num_comp ==…
S. Salman
  • 590
  • 1
  • 6
  • 22
0
votes
1 answer

Krping module for RDMA

I have been trying to make krpring module work which can be found here: http://git.openfabrics.org/?p=~sgrimberg/krping.git;a=summary [ 446.242534] rdma_krping: Unknown symbol ib_alloc_fast_reg_page_list (err -22) [ 446.242542] rdma_krping:…
S. Salman
  • 590
  • 1
  • 6
  • 22
0
votes
1 answer

How to avoid getting stuck in rdma_get_recv_comp() or __ibv_get_cq_event()?

Fellow RDMA hackers, does anyone know if rdma_get_recv_comp(), which calls __ibv_get_cq_event() ever time out? My problem is with the same programs as shown here: RDMA program randomly hangs It works fine, but it's not robust against random client…
Yale Zhang
  • 1,447
  • 12
  • 30
0
votes
1 answer

How are lanes managed on PCIe 3.0 controller embedded on recent Xeon processors?

I'm using several PCIe 3.0 extension cards (GPUs and Infiniband interconnects). I'm wondering how lanes are actually managed and if I may optimize my devices by changing ports or by using some adapters (16x -> 8x). Intel Haswell-EP may manage 40…
jyvet
  • 2,021
  • 15
  • 22
0
votes
1 answer

RDMA WRITE: ibv_wc.byte_len does not match what I try to write

I'm an Infiniband/RDMA newbie playing with RDMA on Mellanox Infiniband hardware. I used the source code here. The program runs pretty normal at first glance: The server writes some bytes to the client memory using RDMA_WRITE operation. However, I…
Weijia Song
  • 153
  • 8
0
votes
1 answer

libibverbs Hello World

I'm trying to find a short "Hello World" example that uses Infiniband with libibverbs. This goes into a good direction, but writing "Hello World" to the terminal should be possible in less than 800 lines. I was hoping for something like this.
mrks
  • 8,033
  • 1
  • 33
  • 62
0
votes
1 answer

Poll system call - is this infiniband communication?

I have a place in my openmpi (compiled for infiniband usage) code where there is no progress output and strace on one of the processes shows this: poll([{fd=5, events=POLLIN}, {fd=14, events=POLLIN}, {fd=23, events=POLLIN}], 3, 0) = 0…
drjrm3
  • 4,474
  • 10
  • 53
  • 91
0
votes
1 answer

rdma connection manager driver pattern

I'm using the OFED 3.18r2 implementation of Infiniband drivers for my application. In particular I'm using the rdma connection manager wrapper functions. To understand better what's going on under the hood I'm used to look at the source code. Doing…
Antonio
  • 35
  • 4
0
votes
1 answer

MPI_Finalize not behaving correctly, orphaned processes

I have an MPI program that is fairly straight forward, essentially "initialize, 2 sends from master to slaves, 2 receives on slaves, do a bunch of system calls for copying/pasting then running a code, tidy up and mpi finalize". This seems…
jackd
  • 337
  • 3
  • 6
  • 14
0
votes
1 answer

Lookup GID/GUID of remote infiniband client

I have an application written in C that opens a TCP connection from the client to the server via IPoIB. I need to look up the GUID/GID of the client from the server, preferably without invoking an external application. Is there a way of doing this…
SP00kY
  • 1
  • 3
0
votes
1 answer

Are Windows IND & Linux verbs implementations compatible?

I've been having a very confusing discussion with a colleague about the use of Network Direct in Windows to provide RDMA, and Linux where we have a working implementation using the verbs api and rdmacm. Said collegue developed his Network Direct…
hookenz
  • 36,432
  • 45
  • 177
  • 286
0
votes
2 answers

What are InfiniBand-Stacks?

I would like to ask you for an explanation what are the "InfiniBand-Stacks". Those were recently changed on our machine and I started running into MPI communication failures. I need some information in order to understand how this might be affecting…
Alexander Cska
  • 738
  • 1
  • 7
  • 29
1 2 3
…
11
12