Stack Overflow
Stack Overflow

Questions tagged [infiniband]

InfiniBand is a high-speed switched fabric communications link technology used in high-performance computing and enterprise data centers.

InfiniBand is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include scalability, high throughput, low latency, quality of service and failover. The InfiniBand architecture specification defines a connection between processor nodes and high performance I/O nodes such as storage devices. Infiniband host bus adapters and network switches are commonly manufactured by both Mellanox and Intel.

178 questions
0
votes
2 answers

Infiniband transport Layer

I am having trouble understanding the context from http://www.redbooks.ibm.com/redbooks/pdfs/sg247351.pdf It is describing how the transport layer is implemented but every info I have read only explains the features not how the transport layer is…
Sungho Hong
  • 340
  • 2
  • 16
0
votes
1 answer

Tensorflow failed to use RDMA on Infiniband network

Executing: python tf_cnn_benchmarks.py \ --local_parameter_device=gpu \ --num_gpus=1 \ --batch_size=2 \ --model=alexnet \ --variable_update=distributed_replicated \ --job_name=ps \ --ps_hosts=192.168.230.107:50000 \ …
William Zhou
  • 1
  • 2
0
votes
1 answer

`ibv_post_send` atomic operations & completion queue

When an atomic operation (either compare-and-swap or fetch-and-add) is posted to an InfiniBand queue, when does its completion event come out? Is the result put in the completion queue immediately after the command has been sent to the other side,…
iAdjunct
  • 2,739
  • 1
  • 18
  • 27
0
votes
1 answer

How to properly understand the IMB-benchmark result

Hello currently I am using Infiniband and testing the performance with IMB-benchmark, I'am currently testing the parallel transfer test and was wondering the results indeed reflect the parallel performance of the 8 processes. The explanation of…
Sungho Hong
  • 340
  • 2
  • 16
0
votes
1 answer

Infiniband support on FreeBSD 11.0

I am trying to bring up a FreeBSD 11.0 server having Infiniband support. From what I know FreeBSD 11.0 already have this support. But I am not seeing any ibv related commands that I can use to run the iWARP traffic. Can anyone suggest what should I…
Shminderjit Singh
  • 31
  • 7
0
votes
1 answer

NVMoF: how to tell a command capsule from a response capsule

Let's say I have RoCEv2 traffic between 2 nodes A & B. A uses RDMA_SEND to send a command capsule to B. Then A receives a capsule from B (B also uses RDMA_SEND). At this moment, how does A know if this is the response to its command sent in step…
Yao Lin
  • 11
  • 2
0
votes
0 answers

How to invoke InfiniBand with OpenMPI

I would like to know how to invoke InfiniBand hardware on CentOS 6x cluster with OpenMPI (static libs.) for running my C++ code. This is how I compile and run: /usr/local/open-mpi/1.10.7/bin/mpic++ -L/usr/local/open-mpi/1.10.7/lib -Bstatic main.cpp…
Boki
  • 193
  • 1
  • 3
  • 15
0
votes
1 answer

MVAPICH 2.3 configure for multiple devices

While mvapich from version 2.3 deprecated the Nemesis interface, is there any way now to configure it at once for Infiniband support with fallback to TCP when failed? Or do I have to have two compilations for different network setups in my grid?
Houmles
  • 197
  • 11
0
votes
0 answers

connect infiniband with intel mpi or other mpi(like openMPI)

I'm new about how to build a infiniband network, and i have not find some useful url about how to do it in windows OS. I wanna to do that need to connect 5~8 servers with infiniband network in windows OS, and i choose the intel mpi for IB and wanna…
Doc.Q
  • 3
  • 4
0
votes
0 answers

How can DAPL offer more functionality than OFA does if DAPL relies solely on OFA as the only layer beneath it?

In my understanding if a system that only has underlying Infiniband connectivity (i.e. not iWarp or anything else which DAPL could use as an alternative) then DAPL exists solely as an abstracted layer on top of OFA/Infiniband. If this is the case,…
Brayme Guaman
  • 175
  • 2
  • 12
0
votes
1 answer

Apache Ignite and Infiniband

I am using Apache Ignite on a cluster that has both 10G Ethernet and QDR Infiniband. I would like to have Ignite use IP over Infiniband (IPoIB) to get the best network performance. To that end, I have put the IP addresses for the IB connections in…
Sumit
  • 113
  • 8
0
votes
0 answers

PCI rescan causes infiniband packet drops

I'm experiencing some issues with the network when I trigger a PCI rescan on Linux with echo 1 > /sys/bus/pci/rescan. I observe data loss, sometimes deadlocks in client/server applications or processes turning into zombie processes. This happens on…
jyvet
  • 2,021
  • 15
  • 22
0
votes
1 answer

Segmentation Fault program compiled into shared library but not static

This may be a somewhat meta question as I think my issue stems from a lack of knowledge about shared vs static libraries. I'm working on a server/client system using infiniband as a learning exercise/prepping for an upcoming work program. Originally…
Nicholas Callahan
  • 153
  • 1
  • 11
0
votes
1 answer

What is a Producer index (PI) in the context of ibv_exp_post_send and ibv_exp_post_task?

I am trying to use the Cross-Channel Communication support described in Appendix D to the RDMA Aware Programming User Manual. Unfortunately I am a bit confused as to the meanings of certain function arguments. My Question The ibv_exp_post_send()…
fintelia
  • 1,201
  • 6
  • 17
0
votes
1 answer

Error using verbs Memory Windows (ibv_alloc_mw)

I am trying to use memory windows and I am getting EPERM (errno=1) when calling ibv_alloc_mw (with both types of MWs). I have mellanox ConnectX-3 cards and the following OFED: ofed_info | head -n 1 MLNX_OFED_LINUX-3.2-2.0.0.0 (OFED-3.2-2.0.0): It…
JC1
  • 657
  • 6
  • 21
1 2 3
…
11 12