Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [infiniband]

InfiniBand is a switched fabric computer network communications link used in high-performance computing and enterprise data centers. This tag should be used for questions about IB related hardware and software.

InfiniBand(IB) is a switched fabric computer network communications link used in high-performance computing and enterprise data centers.

Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable. The InfiniBand architecture specification defines a connection between processor nodes and high performance I/O nodes such as storage devices. InfiniBand host bus adapters and network switches are manufactured by Mellanox and Intel.

In addition to hardware manufactured by Mellanox and Intel there are a variety of software stacks available for the configuration and management of IB fabrics. Most major Linux distributions have some built-in support for Infiniband technologies, and the Open Fabrics Alliance(OFA) maintains the Open Fabrics Enterprise Distribution(OFED) which provides many software and firmware packages for the management, maintenance and setup of IB networks. In addition to the 'official' OFED maintained by the OFA, Mellanox and Intel both produce software stacks for their products that typically include their own customized versions of OFED.

112 questions
1
vote
0 answers

Setting up Infiniband network - wrong route

So I'm trying to set up an Infiniband network alongside my Ethernet network. I have 10 compute nodes and one conductor node. All 11 machines are running CentOS and are plugged in to an Infiniband switch and an Ethernet switch. Ethernet:…
ostrich
  • 71
  • 1
  • 8
1
vote
1 answer

InfiniBand RDMA latency test on Xen's dom0 crashes

The short story: while setting up InfiniBand connection between two servers, I cannot complete the RDMA latency test. It crashes even with breaking the ssh connection. The long story. The first server is the Xen 4.4 with Ubuntu 14.04 as dom0…
Grigory
  • 167
  • 1
  • 2
  • 10
1
vote
0 answers

HP ConnectX cards identifying as ConnectX EN

I bought a lot of four HP branded ConnectX card on Ebay. The cards are : HP InfiniBand 4X DDR ConnectX PCI-E Gen2 Dual-Port HCA 483514-B2 My problem is that two out of four cards identify themselves as ConnectX EN cards. This one seems ok: 04:00.0…
user228840
  • 11
  • 3
1
vote
1 answer

How do I get IP packats forwarded/routed to/from my Infiniband network

I have two networks. One is a standard Ethernet network running IP. The second network is Infiniband which in addition to some custom protocols can talks IPoIB (IP over Infiniband). The router that sits on this network can ping both the Infiniband…
hookenz
  • 14,472
  • 23
  • 88
  • 143
1
vote
2 answers

Alternatives to bridging infiniband ipoib within xen domains?

I've installed xen-hypervisor-4.1 package onto an Ubuntu 12.04.3 server. It has an infiniband adaptor installed and I run IPoIB with it. What I'm wanting to do is provide the ipoib to one or more guests. However using linux bridge is not…
hookenz
  • 14,472
  • 23
  • 88
  • 143
1
vote
3 answers

are DDR infiniband and 10Gbase-CX4 cables the same?

They looks the same. It's SFF-8470 Is it ok to buy a "10Gbase-CX4" cable for an infiniband network ?
user1219721
  • 487
  • 1
  • 6
  • 15
1
vote
2 answers

How to simulate high video payload to benchmark an SSD?

I'm trying to evaluate some SSDs for a video on-demand use-case. We've done some benchmarking tests on them, but we'd like to get an idea of the number of video streams that they can support with a more realistic load test than typical benchmark…
Timothée Boucher
  • 113
  • 3
1
vote
1 answer

NFS performance trouble: with infiniband gets stacked with many users?

I have a RAID system on A server and similar configuration on B server-both with Scientific Linux 5.5. They are connected with infiniband and RAIDS from A are mounted with NFS to ServerB. Now when the users are accessing A-Raids via B-Server, the…
Arman
  • 575
  • 2
  • 8
  • 18
1
vote
1 answer

Infiniband QDR to DDR

I have a bunch of computer with onboard Mellanox ConnectX-2 QDR Infiniband 40Gbps Controllers. They have QSFP connectors. I have a switch with 24 4x CX4 DDR connectors. If I buy QSFP to CX4 cables, will the QDR controllers on the computers be able…
Realn0whereman
  • 153
  • 4
1
vote
1 answer

How can I use nmap against an IPoIB network?

I'm trying to use nmap to check which services are running on a given network but it cannot start just because nmap issues ARP Ping Scans on IPv4 network, and on a Infiniband Network, as far as I know, there's no ARP, because broadcasting is not…
Vinícius Ferrão
  • 5,520
  • 11
  • 55
  • 95
1
vote
1 answer

HPC master node no infiniband network influence on compute nodes - Slurm management

I'm writing because I'm facing an issue that I cannot solve trying to configure a cluster with a master node ( or Frontend node ) as a Virtual machine managing nodes with infiniband network. I use slurm on this nodes, the frontend node is the slurm…
SimoneM
  • 121
  • 1
1
vote
1 answer

Infiniband OpenSM N-to-N port routing configuration

I have 10 servers with two CPUs each and one Mellanox 100G Infiniband NIC per CPU. Each NIC is connected to a single Mellanox 36 port 100G IB switch. My RDMA application runs as one process per NUMA node and binds to the local NIC to avoid cross CPU…
Hugo Maxwell
  • 121
  • 4
1
vote
1 answer

Latency of memory accesses via interconnectors

I'm trying to compare latencies of different node interconnects for a cluster. The goal is to minimize the memory access latency. I have obtained some benchmarks regarding one of the hardware implementations of NUMA architecture with many CPUs. This…
Piotr M
  • 33
  • 3
1
vote
0 answers

what is intel omnipath topology error?

I have lustre setup and connected with intel omnipath. the metadata server is getting rebooted often. in logs, i can see the opa topology error. what is this error means?. will it cause the server to reboot automatically ?
heyloitsmeraj
  • 11
  • 1
1
vote
2 answers

revive an old QLogic IBA7322 Infiniband card on CentOS8.2

I have a few HP Blades Gen7 equipped with QLogic Infiniband cards IBA7322 which I would like to use with CentOS 8. The problem I have is that I cannot find the right drivers for them. All the information I find is either old, or the links don't work…
lucian
  • 131
  • 1
  • 7
1 2 3 4
5
6 7 8