2

I need to check if for an given ipoib IP address the associated IB node is connected to the infiniband switch.

The idea is to get the guid of the IB port to which the IP packets are send and check with a command like ibnodes if the port is connected/reachable.

Anybody have an idea how to realize this in userspace C or on the cmdline? Specially I'm interested to resolve the IP to an guid.

thanks

fho
  • 470
  • 5
  • 12

1 Answers1

4

The answer here is a bit tricky, and I'm afraid that what you want to do is not really possible. But I'm not sure exactly what you really want to do, so let me explain what can be done.

The way that IPoIB resolves IP addresses to GIDs is via ARP -- Ethernet uses broadcast for ARP, while IPoIB uses InfiniBand multicast, but in both cases the ARP request is sent to all nodes in the IP subnet. The actual owner of the address responds with its link-level address, which in the case of IPoIB is the QP number (you don't care about that) and the GID.

For example, on a system that can reach 192.168.1.3 over IPoIB, the ip neigh command shows:

# ip neigh
192.168.1.3 dev ib0 lladdr 80:00:04:04:fe:80:00:00:00:00:00:00:00:02:c9:02:00:21:70:d1 REACHABLE

where the last 16 bytes of that lladdr are the GID; on the remote system that is configured with 192.168.1.3, you can see the GID:

# cat /sys/class/infiniband/mthca0/ports/1/gids/0 
fe80:0000:0000:0000:0002:c902:0021:70d1

The reason I don't think this helps you is that if the IP address is not reachable then the ARP won't be able to resolve the address, and so you won't be able to get a GID to check. In a sense your question is not really well-posed, because there is no fixed association of IP address to IB port -- you could put any IP address you want on any IPoIB interface, and change it at any time.

Roland
  • 6,227
  • 23
  • 29