NetworkManager bridge is not working

Question

I face the same problem on 2 difference machines: A CentOS 8.1 and an openSUSE Leap 15.2
I want to create a bridge on each host that will be used from libvirtfor bridged networking. On both servers, I use NetworkManager and I created the bridges as:

nmcli connection add type bridge bridge.stp no autoconnect yes con-name suse_virt ifname peter_virt ipv4.addresses 192.168.0.3/24 ipv4.gateway 192.168.0.254 ipv4.dns "192.168.0.1,8.8.8.8"  ipv4.method manual
nmcli connection add type bridge-slave master suse_virt autoconnect yes ifname eth0 con-name suse_virt-slave

The only differences between the CentOS and the openSUSE are the connection and interface names as well as the IPs.

When I start the bridge with nmcli con up suse_virt, I can see this:

ip a 

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master peter_virt state UP group default qlen 1000
    link/ether d0:50:99:17:3f:e6 brd ff:ff:ff:ff:ff:ff
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    link/ether 00:0a:f7:09:a2:1d brd ff:ff:ff:ff:ff:ff
7: suse_virt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether d0:50:99:17:3f:e6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.3/24 brd 192.168.0.255 scope global noprefixroute peter_virt
       valid_lft forever preferred_lft forever
    inet6 fe80::af0c:23de:5e0c:ded8/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

Which looks fine, but, only a few seconds later, my whole network is not working! I was suspecting Spanning Tree, but STP is disabled and my switch is an un-managed one. The only solution I have is to stop the bridge and enable the simple connection on each machine.

Restarting Network manager results to even stranger issues like the following:

ping 192.168.0.6
PING 192.168.0.6 (192.168.0.6) 56(84) bytes of data.
64 bytes from 192.168.0.6: icmp_seq=1 ttl=64 time=1.84 ms
^C
--- 192.168.0.6 ping statistics ---
5 packets transmitted, 1 received, 80% packet loss, time 4039ms
rtt min/avg/max/mdev = 1.846/1.846/1.846/0.000 ms

ping -c4 192.168.0.254
PING 192.168.0.254 (192.168.0.254) 56(84) bytes of data.
From 192.168.0.3 icmp_seq=1 Destination Host Unreachable
From 192.168.0.3 icmp_seq=2 Destination Host Unreachable
From 192.168.0.3 icmp_seq=3 Destination Host Unreachable
From 192.168.0.3 icmp_seq=4 Destination Host Unreachable

--- 192.168.0.254 ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 3071ms

The configuration as reported from the NM is:

nmcli con show suse_virt

connection.id:                          suse_virt
connection.uuid:                        45bc9ba8-8120-4bc5-93f4-168f28687f88
connection.stable-id:                   --
connection.type:                        bridge
connection.interface-name:              suse_virt
connection.autoconnect:                 yes
connection.autoconnect-priority:        0
connection.autoconnect-retries:         -1 (default)
connection.multi-connect:               0 (default)
connection.auth-retries:                -1
connection.timestamp:                   1600532118
connection.read-only:                   no
connection.permissions:                 --
connection.zone:                        --
connection.master:                      --
connection.slave-type:                  --
connection.autoconnect-slaves:          -1 (default)
connection.secondaries:                 --
connection.gateway-ping-timeout:        0
connection.metered:                     unknown
connection.lldp:                        default
connection.mdns:                        -1 (default)
connection.llmnr:                       -1 (default)
connection.wait-device-timeout:         -1
ipv4.method:                            manual
ipv4.dns:                               192.168.0.1,8.8.8.8
ipv4.dns-search:                        --
ipv4.dns-options:                       --
ipv4.dns-priority:                      0
ipv4.addresses:                         192.168.0.3/24
ipv4.gateway:                           192.168.0.254
ipv4.routes:                            --
ipv4.route-metric:                      -1
ipv4.route-table:                       0 (unspec)
ipv4.routing-rules:                     --
ipv4.ignore-auto-routes:                no
ipv4.ignore-auto-dns:                   no
ipv4.dhcp-client-id:                    --
ipv4.dhcp-iaid:                         --
ipv4.dhcp-timeout:                      0 (default)
ipv4.dhcp-send-hostname:                yes
ipv4.dhcp-hostname:                     --
ipv4.dhcp-fqdn:                         --
ipv4.dhcp-hostname-flags:               0x0 (none)
ipv4.never-default:                     no
ipv4.may-fail:                          yes
ipv4.dad-timeout:                       -1 (default)
ipv6.method:                            auto
ipv6.dns:                               --
ipv6.dns-search:                        --
ipv6.dns-options:                       --
ipv6.dns-priority:                      0
ipv6.addresses:                         --
ipv6.gateway:                           --
ipv6.routes:                            --
ipv6.route-metric:                      -1
ipv6.route-table:                       0 (unspec)
ipv6.routing-rules:                     --
ipv6.ignore-auto-routes:                no
ipv6.ignore-auto-dns:                   no
ipv6.never-default:                     no
ipv6.may-fail:                          yes
ipv6.ip6-privacy:                       -1 (unknown)
ipv6.addr-gen-mode:                     stable-privacy
ipv6.ra-timeout:                        0 (default)
ipv6.dhcp-duid:                         --
ipv6.dhcp-iaid:                         --
ipv6.dhcp-timeout:                      0 (default)
ipv6.dhcp-send-hostname:                yes
ipv6.dhcp-hostname:                     --
ipv6.dhcp-hostname-flags:               0x0 (none)
ipv6.token:                             --
bridge.mac-address:                     D0:50:99:17:3F:E6
bridge.stp:                             no
bridge.priority:                        128
bridge.forward-delay:                   15
bridge.hello-time:                      2
bridge.max-age:                         20
bridge.ageing-time:                     300
bridge.group-forward-mask:              0
bridge.multicast-snooping:              yes
bridge.vlan-filtering:                  no
bridge.vlan-default-pvid:               1
bridge.vlans:                           --
proxy.method:                           none
proxy.browser-only:                     no
proxy.pac-url:                          --
proxy.pac-script:                       --
GENERAL.NAME:                           suse_virt
GENERAL.UUID:                           45bc9ba8-8120-4bc5-93f4-168f28687f88
GENERAL.DEVICES:                        suse_virt
GENERAL.IP-IFACE:                       suse_virt
GENERAL.STATE:                          activated
GENERAL.DEFAULT:                        yes
GENERAL.DEFAULT6:                       no
GENERAL.SPEC-OBJECT:                    --
GENERAL.VPN:                            no
GENERAL.DBUS-PATH:                      /org/freedesktop/NetworkManager/ActiveConnection/3
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/Settings/7
bridge.priority:                        128
bridge.forward-delay:                   15
bridge.hello-time:                      2
bridge.max-age:                         20
bridge.ageing-time:                     300
bridge.group-forward-mask:              0
bridge.multicast-snooping:              yes
bridge.vlan-filtering:                  no
bridge.vlan-default-pvid:               1
bridge.vlans:                           --
proxy.method:                           none
proxy.browser-only:                     no
proxy.pac-url:                          --
proxy.pac-script:                       --
GENERAL.NAME:                           suse_virt
GENERAL.UUID:                           45bc9ba8-8120-4bc5-93f4-168f28687f88
GENERAL.DEVICES:                        suse_virt
GENERAL.IP-IFACE:                       suse_virt
GENERAL.STATE:                          activated
GENERAL.DEFAULT:                        yes
GENERAL.DEFAULT6:                       no
GENERAL.SPEC-OBJECT:                    --
GENERAL.VPN:                            no
GENERAL.DBUS-PATH:                      /org/freedesktop/NetworkManager/ActiveConnection/3
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/Settings/7
GENERAL.ZONE:                           --
GENERAL.MASTER-PATH:                    --
IP4.ADDRESS[1]:                         192.168.0.3/24
IP4.GATEWAY:                            192.168.0.254
IP4.ROUTE[1]:                           dst = 192.168.0.0/24, nh = 0.0.0.0, mt = 425
IP4.ROUTE[2]:                           dst = 0.0.0.0/0, nh = 192.168.0.254, mt = 20425
IP4.DNS[1]:                             192.168.0.1
IP4.DNS[2]:                             8.8.8.8

nmcli con show suse_virt-slave

connection.id:                          suse_virt-slave
connection.uuid:                        8a6be03b-debc-472d-a44d-eac7145b6ae0
connection.stable-id:                   --
connection.type:                        802-3-ethernet
connection.interface-name:              eth0
connection.autoconnect:                 yes
connection.autoconnect-priority:        0
connection.autoconnect-retries:         -1 (default)
connection.multi-connect:               0 (default)
connection.auth-retries:                -1
connection.timestamp:                   1600532164
connection.read-only:                   no
connection.permissions:                 --
connection.zone:                        --
connection.master:                      suse_virt
connection.slave-type:                  bridge
connection.autoconnect-slaves:          -1 (default)
connection.secondaries:                 --
connection.gateway-ping-timeout:        0
connection.metered:                     unknown
connection.lldp:                        default
connection.mdns:                        -1 (default)
connection.llmnr:                       -1 (default)
connection.wait-device-timeout:         -1
802-3-ethernet.port:                    --
802-3-ethernet.speed:                   0
802-3-ethernet.duplex:                  --
802-3-ethernet.auto-negotiate:          no
802-3-ethernet.mac-address:             --
802-3-ethernet.cloned-mac-address:      --
802-3-ethernet.generate-mac-address-mask:--
802-3-ethernet.mac-address-blacklist:   --
802-3-ethernet.mtu:                     auto
802-3-ethernet.s390-subchannels:        --
802-3-ethernet.s390-nettype:            --
802-3-ethernet.s390-options:            --
802-3-ethernet.wake-on-lan:             default
802-3-ethernet.wake-on-lan-password:    --
bridge-port.priority:                   32
bridge-port.path-cost:                  100
bridge-port.hairpin-mode:               yes
bridge-port.vlans:                      --
GENERAL.NAME:                           suse_virt-slave
GENERAL.UUID:                           8a6be03b-debc-472d-a44d-eac7145b6ae0
GENERAL.DEVICES:                        eth0
GENERAL.IP-IFACE:                       eth0
GENERAL.STATE:                          activated
GENERAL.DEFAULT:                        no
GENERAL.DEFAULT6:                       no
GENERAL.SPEC-OBJECT:                    --
GENERAL.VPN:                            no
GENERAL.DBUS-PATH:                      /org/freedesktop/NetworkManager/ActiveConnection/4
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/Settings/2
GENERAL.ZONE:                           --
GENERAL.MASTER-PATH:                    /org/freedesktop/NetworkManager/Devices/5
IP4.GATEWAY:                            --
IP6.GATEWAY:                            --

ip route:

default via 192.168.0.254 dev peter_virt proto static metric 425 
192.168.0.0/24 dev peter_virt proto kernel scope link src 192.168.0.3 metric 425

ip -ts -4 monitor

[2020-09-20T07:17:55.876194] 192.168.0.1 dev suse_virt lladdr a8:a1:59:00:35:88 STALE
       valid_lft forever preferred_lft forever
[2020-09-20T07:18:01.252200] 192.168.0.1 dev suse_virt lladdr a8:a1:59:00:35:88 PROBE
[2020-09-20T07:18:04.324192] 192.168.0.1 dev suse_virt  FAILED
       valid_lft forever preferred_lft forever
[2020-09-20T07:18:06.180909] 192.168.0.1 dev suse_virt lladdr a8:a1:59:00:35:88 REACHABLE
       valid_lft forever preferred_lft forever
       valid_lft forever preferred_lft forever
       valid_lft forever preferred_lft forever
       valid_lft forever preferred_lft forever
       valid_lft forever preferred_lft forever
[2020-09-20T07:18:45.032169] 192.168.0.1 dev suse_virt lladdr a8:a1:59:00:35:88 STALE
       valid_lft forever preferred_lft forever
       valid_lft forever preferred_lft forever
       valid_lft forever preferred_lft forever
       valid_lft forever preferred_lft forever
[2020-09-20T07:18:56.292191] 192.168.0.1 dev suse_virt lladdr a8:a1:59:00:35:88 PROBE
       valid_lft forever preferred_lft forever
[2020-09-20T07:18:59.364174] 192.168.0.1 dev suse_virt  FAILED
[2020-09-20T07:19:01.224802] 192.168.0.1 dev suse_virt lladdr a8:a1:59:00:35:88 REACHABLE
[2020-09-20T07:19:07.556221] 192.168.0.254 dev suse_virt lladdr 04:bf:6d:7f:35:14 STALE
       valid_lft forever preferred_lft forever

you can run `ip -ts -4 monitor` to follow what's going on, in case an unexpected change happens. Also can you provide the routes (`ip route`)? — A.B, Sep 19 '20 at 19:30
route is OK, of course: default via 192.168.0.254 dev peter_virt proto static metric 425 192.168.0.0/24 dev peter_virt proto kernel scope link src 192.168.0.3 metric 42 @A.B The `ip -ts -4 monitor` gave me some strange results, I added in the original post. — ptselios, Sep 20 '20 at 04:28
I know this would be huge, but it appears you filtered the result of ip monitor. So if something caused the ARP request failures in what isn't displayed, it can't be known (but I guess there was nothing useful?). Did you also try disabling any sort of firewall, and also any software working with network, like libvirt itself, Docker etc. to see if there's any difference? Are you also sure there's no duplicate IP address or MAC address (affects upstream switches) on the network? It's all about debugging to find any anomaly anywhere. tcpdump could also help etc. — A.B, Sep 20 '20 at 07:40
I know @A.B. I greped the IPv6 related information. My switch is a dump TP Link, BTW. Libvirt is not working, exactly because I wanted to avoid any issues with other bridges. Can you see anything that could guide me to the possible solution? I have triple checked the MACs and I even removed them from the ifcfg files in order to be sure. No difference. For me the behavior of the LAN seems like there is a problem with the way the bridge is propagated. When I enable the switch, the whole LAN is rendered useless. So loop? I will try to isolate ports on the switch. — ptselios, Sep 20 '20 at 08:24
I will try to isolate the 2 PCs and see if there is a global issue or not. — ptselios, Sep 20 '20 at 14:35
Disabling STP is _not_ a good idea as you can create a broadcast storm that will bring down the entire network. — Ron Maupin, Sep 20 '20 at 16:03
For the time being, I need to find a working bridge configuration. Then, STP will be enabled. — ptselios, Sep 22 '20 at 08:01

score 0 · Answer 1 · answered Sep 19 '20 at 16:48

0

Would you have by any chance two connections on eth0? You can check with nmcli con show to see the list of connections. The only connection on eth0 should be your suse_virt-slave connection.

answered Sep 19 '20 at 16:48

Alex

46
4

Of course not.The old connection is removed. And this doesn't explain why the WHOLE network (LAN) is not working. – ptselios Sep 19 '20 at 18:43

score 0 · Answer 2 · answered Dec 29 '22 at 12:30

0

I ran into the same issue, I think, I followed this guide

which takes one through creating a libvirt network using a bridge interface and then using that network as a regular network on the domain. This works well for me. I had some issues which the bridge breaking all networking when it probably shouldn't have but once everything was up, it worked for me.

answered Dec 29 '22 at 12:30

CamW

151
5

Well, I followed the similar guide from the official livbirt/kvm/qemu/whaterver site. But in RHEL 8 there are no iptables, they are replaced by nft. And even when I replaced it with iptables, the result was the same. Only when I stop firewalld everything is nice. – ptselios Jan 08 '23 at 09:04

NetworkManager bridge is not working

2 Answers2