1

[Resolved]: The network cable had been connected to the integrated NIC physical interface, which although accessible, was no longer enabled. The problem was solved by connecting the cable to the additional I350 Gigabit adapter. Stupid as it sounds, as usual, a human mistake. Server running smooth and pleasantly.

Many thanks in advance for whoever comes up with an idea for this puzzle!

Please excuse the typing errors in the terminal output, but I can't dump from the iDRAC virtual console. I wish, then I wouldn't have a network problem...

The story short: I am setting up a new Dell EMC PowerEdge. Ubuntu 16.04.3 LTS installation has ran smoothly using iDRAC v8 with self-configured network profile.

Now, I try to bring up the additional Gigabit Ethernet interfaces (2xIntel I350) and I can't even see the device locally.

ifconfig shows me a fully configured interface (enp177s0f0) and ifup confirms this with RTNETLINK enp177s0f0 already configured.

Output of ifconfig:

ifconfig -a

    enp177s0f0 Link encap: Ethernet HWaddr xx:xx:xx:xx:xx:xx
    inet addr: 137.193.xx.aa Bcast: 137.193.xx.255 Mask: 255.255.255.0
    UP BROADCAST MULTICAT MTU:1500 Metric:1
    RX packets: 0 errors:0 dropped:0 overruns:0 frame:0
    TX packets: 0 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
    Memory: ....

    enp177s0f1 Link encap: Ethernet HWaddr xx:xx:xx:xx:xx:xx
    BROADCAST MULTICAT MTU:1500 Metric:1
    RX packets: 0 errors:0 dropped:0 overruns:0 frame:0
    TX packets: 0 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
    Memory: ...

    lo Link encap: Local Loopback
    inet addr:127.0.0.1 Mask:255.0.0.0
    inet6 addr: ::1/128 Scope:Host
    UP LOOPBACK RUNNING MTU:65636 Metric:1
    RX packets:399 errors:0 dropped:0 overruns:0 frame:0
    TX packets: 399 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1
    RX bytes:33576 (33.5 KB) TX bytes: 33576 (33.5 KB)

    virbr0 Link encap: Ethernet HWaddr yy:yy:yy:yy:yy:yy
    inet addrs: 137.193.xx.aa Bcast 137.193.xx.255 Mask: 255.255.255.0
    UP BROADCAST MULTICAST MTU:1500 Metric:1
    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
    TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:0 Tx bytes: 180

    virbr0-nic Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy
    BROADCAST MULTICAST MTU:1500 Metric:1
    RX packets:0 errors:0 dropped:0 overruns:0 frame:0
    TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:0 Tx bytes: 0

However RTNETLINK ip link show dev enp177s0f0 shows me the same interface in state "down". Trying to bring the link up (ip link set dev enp177s0f0 up) is doing literally nothing.

I have restarted, stopped and started the networking interfaces (/etc/init.d/networking restart) in all possible combinations. Reboots included. The /etc/network/interfaces contains a valid network configuration working on other servers.

The output of ethtool enp177s0f0 tells the link cannot be detected:

output of ethtool

Any ideas what could be happening? Anyone knows if Ubuntu may be lacking drivers for the I350 and if so, how to bring them on the server through iDRAC?

Could it be a speed negotiation issue with the switch? I am using an AT FS705EFC/SC with cat6a network cable.

I use a different IP address on the iDRAC and the Gigabit interfaces. To my knowledge, this isn't an issue, but I am happy to learn if I am missing something here.

In the "system network devices" tab of the idrac web interface, the following is listed for the i350 card: idrac listing for i350

Ela
  • 11
  • 3
  • 1
    Please insert the output of ifconfig into your question, so the question will still be valid when the remote image goes away. – Andrew Schulman Jan 17 '18 at 14:47
  • It would be nice, and I understand your point. But like I explained, from the iDRAC remote console I can't dump to file neither copy to the host. So, before I start hand-typing MAC addresses, I rather wait until someone comes with a hint. – Ela Jan 17 '18 at 14:52
  • Could you please confirm that the kernel module for i350 is loaded? `modprobe igb` then `lsmod`. This is not a a problem with your switch ot the links being down. – sfk Jan 18 '18 at 08:19
  • The `modprobe -v igb` has no output. With `lsmod`, igb shows up (size 196608) used by 0. – Ela Jan 18 '18 at 11:39
  • The i350 is an add-on pcie card or is it the onboard NDC (note: based on the ifconfig screenshot, it seem that your server has no em1/em2 interfaces thus i guess it is the latter case)? Also, is it visible with `lspci`, is it visible using the idrac (in the inventory section) ? – sfk Jan 18 '18 at 13:47
  • The i350 are add-ons. In the inventory, the following interfaces appear (4x): 2x `Intel(R) Gigabit 2P I350-t Adapter - B4:96:91:17:02:21` 2x `Broadcom Adv. Dual 10GBASE-T Ethernet - 50:9A:4C:AA:A9:1E` Could it be that I messed up in the hardware configuration? It's my first time doing this end-to-end... – Ela Jan 23 '18 at 10:41
  • Not 100% sure, but interface enp177s0f0 is probably the "OS-to-idrac" virtual USB NIC. Not a regular Ethernet card. Could you please run `ifconfig -a` to verify that there are no other empxxxxx interfaces in your system? – sfk Jan 24 '18 at 11:24
  • Indeed, there exist `enp177s0f0` and `enp177s0f1`, plus the `virbr` and `virb-nic` which I had mentally associated to the idrac. I typesetted the complete ifconfig -a output in the original question. In one of the tests I had done, I configured the `enp177s0f1` with an equivalent network configuration and brought it up. This didn't get me online either. – Ela Jan 24 '18 at 17:17
  • Ok, so now, based on the hw inventory from idrac, and ifconfig -a output, you can check the HW mac address for each card/port and tell which one you can see in the OS. – sfk Jan 24 '18 at 18:58
  • The mac address of the `enp177s0f0` and `enp177s0f1` (visible in the os with `ifconfig`) correspond to the mac addresses for the i350 cards that I see through idrac. However, these are not listed in the inventory, but in the System/Network Devices tab. The mac addresses are listed twice per card, as mac address and as virtual mac address. – Ela Jan 25 '18 at 10:58
  • Interesting is, I noticed the statement "Family Driver Version: Unavailable" although "OS Driver State: Operational" – Ela Jan 25 '18 at 11:05

0 Answers0