2

My objective is to have all KVM guest VMs send and receive traffic on em2 with addresses on the 192.168.2.0/24 subnet.

I have a host Linux machine (CentOS 7) with several NICs, 2 of which are in use in this scenario, em1 and em2.

The em1 interface has an IP of 192.168.0.131. The em2 interface has been attached to br0, so it doesn't have an IP itself, but br0 has been assigned an IP address of 192.168.2.1.

I have created a route on my Netgear firewall to direct 192.168.2.0/24 traffic to 192.168.2.1 but this address doesn't show as an attached device the way 192.168.0.131 does, maybe because it's a virtual Linux bridge.

From the host VM, I can ping both the "bridge gateway", the VM guest, and the firewall gateway to the internet:

[root@boss ~]# ping -c1 192.168.2.1
64 bytes from 192.168.2.1: icmp_seq=1 ttl=64 time=0.085 ms

[root@boss ~]# ping -c1 192.168.2.10
64 bytes from 192.168.2.10: icmp_seq=1 ttl=64 time=0.476 ms

[root@boss ~]# ping -c1 192.168.0.254
64 bytes from 192.168.0.254: icmp_seq=1 ttl=64 time=4.17 ms

And from the guest VM, I can ping em1, but not the internet gateway, 192.168.0.254:

[root@localhost ~]# ping -c1 192.168.0.131
64 bytes from 192.168.0.131: icmp_seq=1 ttl=64 time=0.282 ms

[root@localhost ~]# ping -c1 192.168.0.254
PING 192.168.0.254 (192.168.0.254) 56(84) bytes of data.

--- 192.168.0.254 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

This is my config for em2:

DEVICE=em2
TYPE=Ethernet
ONBOOT=yes
BRIDGE=br0

And br0:

DEVICE=br0
BOOTPROTO=none
ONBOOT=yes
TYPE=Bridge
IPADDR=192.168.2.1
PREFIX=24
GATEWAY=192.168.0.254
ZONE=public
STP=no

My routing table on the VM host:

[root@boss ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.0.254   0.0.0.0         UG    0      0        0 em1
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 em1
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 br0

The guest VM was started with virt-install:

virt-install \
  --name vm-guest-1 \
  --network bridge=br0 \
  --virt-type kvm \

Guest VM eth0:

DEVICE="eth0"
BOOTPROTO="none"
ONBOOT="yes"
TYPE="Ethernet"
IPADDR="192.168.2.10"
NETMASK=255.255.255.0

And the guest VM routing table:

[root@localhost ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.2.1     0.0.0.0         UG    0      0        0 eth0
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0

As requested, my host bridge output:

[root@boss ~]# brctl show
bridge name     bridge id           STP enabled    interfaces
br0             8000.d4ae529de039   no             em2
                                                   vnet0

Question/Problem:

How do I / why can't I route into my guest VM, or rather, why can't my guest VM get out to the internet?

OnNIX
  • 121
  • 1
  • 3
  • The static route on your Netgear device is wrong: you must route `192.168.2.0/24` through an address the device already knows how to reach: `192.168.0.131`. – Piotr P. Karwasz Dec 01 '19 at 07:26
  • Remark: since you bridged the VM's virtual interface with `em1`, you don't need routing at all. All Ethernet frames sent by the VM are resent on the physical network. You just need to assign the VM an address in `192.168.0.0/24` or use DHCP. – Piotr P. Karwasz Dec 01 '19 at 07:29
  • Your first comment, my Netgear routing table...I've done that, and it works, but then all my `192.168.2.0/24` will route through em1, which I do not want. Also, I have not bridged the VM's interface with `em1`, I'm not sure where you got that from... I also specifically want my VMs on the 192.168.2.0 subnet. Perhaps you've misunderstood my question. – OnNIX Dec 01 '19 at 07:32
  • libvirt probably did that. Add the output of `bridge link show` or `brctl show` on the host machine. – Piotr P. Karwasz Dec 01 '19 at 07:35
  • `brctl show` added – OnNIX Dec 01 '19 at 07:39

1 Answers1

1

You have basically two options:

  1. In your current configuration vnet0 and em2 are bridged, so the VM can reach the Netgear device without routing. You just need to assign the VM an address in 192.168.0.0/24 or use DHCP.

  2. If you need to route, remove the em2 interface from the bridge and assign to it an address in 192.168.0.0/24. Then add a new routing table to /etc/iproute2/rt_tables:

    100 vms

    Now add a routing table selection rule and a default route for your VMs:

    ip route add default dev em2 via 192.168.0.254 table vms
ip rule add iif br0 table vms

    In addition you need to modify the static route on the Netgear device to point to em2's address.

Edit: In the second configuration Ethernet frames from the VMs do not leak on the physical network. The additional routing table is selected only when it needs to route something from the bridge br0 (i.e. the virtual guests) and sends the traffic through em2 instead of em1 (the default route in the main table). You can see all routing tables at once with:

ip route show table all

You can read more about routing tables on the Guide to IP Layer Network Administration with Linux.

Piotr P. Karwasz
  • 5,748
  • 2
  • 11
  • 21
  • My objective is for all guest VM traffic to route through `em2`, on subnet 192.168.2.0/24. Can you help me understand how your answer will accomplish my objectives? – OnNIX Dec 01 '19 at 07:52
  • I added some more information to the answer. Basically you: 1. don't want traffic from the VMs to leak on the wire (so unbridge), 2. you need a way to tell Linux to route the VMs traffic through `em2` instead of `em1`, hence the new routing table just for traffic from the VMs. – Piotr P. Karwasz Dec 01 '19 at 08:02
  • Thank you for your suggestion, I will check this out tomorrow. – OnNIX Dec 01 '19 at 08:07
  • A last remark: the `GATEWAY` in an interface config must be on the same interface and in the same subnet as the `IPADDR`. Delete it from `br0` definition or it might mess up your (main) default route. – Piotr P. Karwasz Dec 01 '19 at 08:10