1

I have a rack with Supermicro servers each with IPMI interface. I am able to connect to any IPMI web/cli interface from my notebook. I am also able to ping any IPMI interface from any Supermicro server except the Supermicro server, where the IPMI interface is.

An example of a better explanation of my problem:
servers A, B, C can ping IPMI interface of server D, but server D can't ping IPMI interface of server D
servers B, C, D can ping IPMI interface of server A, but server A can't ping IPMI interface of server A
and so on

It looks like each IPMI interface discards packets from a NIC from the same server where IPMI is. Does anybody know why? I don't see any settings or restrictions on any IPMI interface.
I need it for SNMP. I want to get info like by ipmitool sdr but over SNMP.

Radek Uhlíř
  • 11
  • 5

1 Answers1

0

That is because supermicro servers uses shared NC-SI connections. NC-SI means network controller sideband interface and it allows to share one physical network interface by operating system running on the host and by BMC. I do not know exact model of your supermicro servers but it is pretty clear it has NC-SI connection like that:

Shared NIC

IPMI interface does not discard any traffic from the server because it simply does not receive that traffic. NC-SI means the physical NIC and LAN connector are shared by BMC and CPU. So what happen when you send a request from the host OS to the BMC in the same server?

  • The host OS knows IP address so it starts with MAC address search via ARP. OS creates an ARP request with a broadcast destination MAC and pass it to the network interface card.
  • Network interface card sends the ARP request to the network switch.
  • The switch forwards the ARP request to all ports except the port where the request was received.
  • Nobody replies to the ARP request because BMC has not received anything.

A few workarounds can be applied:

  • Use IPMI over IPMB instead of IPMI over IP when connecting to local BMC.
  • Assign BMC and server IP addresses from different IP ranges or different VLANs and force L3 routing for the traffic between server and its BMC.
  • Buy a dedicated BMC NIC. Some server models allow that.
Zaboj Campula
  • 3,155
  • 3
  • 24
  • 42