Is it possible to get any kind of unique identifier for the Infiniband switch that a particular Linux box is connected to?

Question

I am trying to identify which Infiniband NIC's are connected to which switches, in a manner that is very similar to this question.

However, the solution proposed there is not working for Mellanox Infiniband switches, and I have not been able to find any alternatives.

Is there a standard way to query for an identifier for the Infiniband switch that a particular machine is connected to?

adiflinux · Answer 1 · 2014-06-19T00:01:57.127

The MLNX_EN for Linux or MTNIC Driver should be installed on your Linux Machine. Than use the 'ethtool' or 'mstflint' commands as follows:

ethtool -i eth2 driver: mlnx_en (MT_0DD0120009_CX3) <--- This is the PSID version: 2.1.6 (Feb 2014) firmware-version: 2.30.8000 bus-info: 0000:1a:00.0 mstflint: 1. Get the device location on the PCI bus:
lspci ....... 04:00.0 InfiniBand: Mellanox Technologies MT4099 [ConnectX-3 VPI FDR, IB (56Gb/s) and 40GigE; PCIe3.0 x8 8GT/s; RoHS R6]. 2. Get the PSID of your product, enter: > mstflint -d 04:00.0 q Image type: FS2 FW Version: 2.30.8000 Device ID: 4099 Chip Revision: A1 Description: Node Port1 Port2 Sys image GUIDs: 0002c90307e25050 0002c90307e25051
MACs: 0002c9e25750 0002c9e25751
VSD: N/A
PSID: MT_1090120019

Following Mellanox support page: http://www.mellanox.com/page/firmware_HCA_FW_identification

While I will certainly try this, it is better to put the exact commands into the answer that you think will resolve the question. — merlin2011, Jun 18 '14 at 23:57

score 0 · Answer 2 · answered Jun 19 '14 at 05:27

0

Perhaps you can use the ibnetdiscover tools from the infiniband-diags package. It prints out the InfiniBand network topology, including the switches and which host port is connected to which switch port.

answered Jun 19 '14 at 05:27

haggai_e

272
1
7

score 0 · Answer 3 · answered Jun 23 '14 at 23:15

You are asking about infiniband, so any ethernet-related tool (as described in the other question that you link to or in one of the answers here) is irrelevant.

The ID of the switch in infiniband is its GUID.

Run ibhosts to get the list of all the HCAs that you have in your subnet. Run ibswitches to get the list of all the switches that you have in your subnet.

Now, if you want to get the complete topology, run ibnetdiscover, as suggested by @haggai_e.

If you question is more specific, like what is the switch GUID that is connected to the 1st port of this NIC (called "HCA" in infiniband) on this particular Linux box, you can directly query this switch with smpquery tool.

Run it as follows: smpquery -D NodeInfo 0,1

Run smpquery -h for more usage info.

Is it possible to get any kind of unique identifier for the Infiniband switch that a particular Linux box is connected to?

3 Answers3