-1

I am seeking a procedure list with examples on how to configure VLAN networking for a CentOS7 KVM host that has two RHEL7 guests. I would post ALL of the myriad of things I have tried, but that would only waste lots of time and space. I am having problems finding the ideal example.

I have the two guests built and working. The host machine has only two ethernet interfaces. We have connected 802.1q trunks to both of them. When I employ traditional ifcfg networking with a ifcfg-em1 and ifcfg-em2 with all info in place and a single gateway in ifcfg-em2, all works out to the internet.

em1 has the private vlans, with the native vlan for host use only, and VLAN 2 and 3 will be the other two private vlans that I want my two guests to use.

em2 has the public vlans, somewhat similar configuration - native vlan (101) for both the host and the first guest, and VLAN 103 for the second guest. I would like the host and the first guest to use different IPs on the same subnet on the native vlan.

I started by trying to build a bridge for the public connections. I removed all the ip address info from the ifcfg's of the physical em interfaces. Then I built a br2 interface in /etc/sysconfig/network-scripts and populated it with an IP address and subnet that would be for the host. There I also set GATEWAY=123.1.2.3 (sanitized IP) as the only gateway in the system, and I have added DEFROUTE=yes. With this set, from the kvm host I can ping the gateway at 123.1.2.3, but nothing beyond. (using 'ping 123.1.2.3 -I br2').

I know I need to build a virtual network in kvm virt-manager under the connections for each guest, but when I do, nothing changes. There the fog starts and I have about 4 ways, none of which work, to try bridges or taps to get the guest to connect to the host bridge.

So it appears I have a basic networking problem at the host where I can't get past the default gateway, and then other problems between the guests and the host where they don't appear to be on the bridge and can't see anything but their own IP addresses.

I am sure someone has configured a scenario like this before, and if they could just give a detailed example broken into steps, I would greatly appreciate it.

sharpy1064
  • 11
  • 3

2 Answers2

1

I figured this out after much labor, so for this not be a dead-end thread, here's the solution I found after a LOT of searching. There was a design change as it eventually seemed more logical to have everything tagged on all interfaces instead of making one of the physical interfaces to be untagged, for utmost future flexibility.

First, you start with the base CentOS load with NO NETWORKING CONFIGURED.
Do not touch /etc/sysconfig/network-scripts/ (yet)

Then, as an example:

Suppose two em interfaces labeled em1 and em2. em1 and em2 are attached to trunk ports on your ethernet switch that supports 802.1q vlan tagging.

From a FRESH INSTALL of the host with no previous network configuration you should only see the two em intefaces with no network configuration.

**** DO NOT EDIT OR SET ANY IFCFG FILE in /etc/sysconfig/network-scripts AT THIS POINT **** Design:

For 2 ports and 4 vlans:
em1 (or first int) outside / vlan 10
em2 (or second int) inside / vlans 20, 30 & 40
vlan 10 (Public Network - 10.0.0.128/27) 
vlan 20 (Private Network - 192.168.1.0/24)
vlan 30 (Private Network - 192.168.2.0/24) 
vlan 40 (Private Network - 192.168.3.0/24)

Create 4 vlan based bridges:
nmcli con add type bridge autoconnect yes con-name virbr10 ifname virbr10 bridge.stp no
nmcli con add type bridge autoconnect yes con-name virbr20 ifname virbr20 bridge.stp no 
nmcli con add type bridge autoconnect yes con-name virbr30 ifname virbr30 bridge.stp no
nmcli con add type bridge autoconnect yes con-name virbr40 ifname virbr40 bridge.stp no
    
Configure the virtual bridges with ip information:
nmcli con modify virbr10 ipv4.addresses 10.0.0.150/27 ipv4.method manual
nmcli con modify virbr20 ipv4.addresses 192.168.1.29/24 ipv4.method manual
nmcli con modify virbr30 ipv4.addresses 192.168.2.60/24 ipv4.method manual
nmcli con modify virbr40 ipv4.addresses 192.168.3.15/24 ipv4.method manua
    
Add a vlan to the bridges:
nmcli con add type vlan autoconnect yes con-name em1.10 dev em1 id 10 master virbr10
nmcli con add type vlan autoconnect yes con-name em2.20 dev em2 id 20 master virbr20
nmcli con add type vlan autoconnect yes con-name em2.30 dev em2 id 30 master virbr30
nmcli con add type vlan autoconnect yes con-name em2.40 dev em2 id 40 master virbr40

Configure hypervisor gateway dns and hostname:
nmcli con modify virbr10 ipv4.gateway 10.0.0.129
nmcli con modify virbr10 ipv4.dns "10.0.0.140 10.0.0.141"
nmcli general hostname myserver@mydomain.com
    
Add a route to get to a inside host if desired:
nmcli connection modify virbr20 +ipv4.routes "10.1.110.0/24 192.168.1.1"

This should create 4 bridge interfaces and 4 vlan interfaces attached to the bridge interfaces.

Turn on the bridges:
nmcli con up virbr10
nmcli con up virbr20
nmcli con up virbr30
nmcli con up virbr40
nmcli con show
    
Use ip command to view the IP settings:
ip a s
    
show more info on the interfaces:
nmcli -p con show virbr10
nmcli -p con show virbr20
nmcli -p con show virbr30
nmcli -p con show virbr40

IMPORTANT: For the base interfaces (em1, em2, etc), there must be some additional changing of parameters in /etc/sysconfig/network-scripts before they will come up. Change any ipv6 parameters set as 'yes' to 'no'. Set BOOTPROTO=none and ONBOOT=yes. Restart networking. If there are any issues, check ifcfg files for onboot and other desired settings. Make sure that all ends up as:

nmcli dev status

DEVICE   TYPE      STATE      CONNECTION 
virbr10  bridge    connected  virbr10    
virbr20  bridge    connected  virbr20
virbr30  bridge    connected  virbr30    
virbr40  bridge    connected  virbr40    
em1      ethernet  connected  em1        
em2      ethernet  connected  em2        
em1.10   vlan      connected  em1.10     
em2.20   vlan      connected  em2.20
em2.30   vlan      connected  em2.30    
em2.40   vlan      connected  em2.40     
lo       loopback  unmanaged  --
sharpy1064
  • 11
  • 3
0

Here is some additional info as it relates to KVM and the virt-manager that is not very obvious and can be tricky when first seen:

When KVM is installed AFTER the bridges are built, when selected for use in the virt-manager interface, they show up in 'Virtual Network Interface' as Network source: Specify shared device name with Bridge name = "virbr10" (whatever was set as the bridge name) and Device model: virtio (always use this device for vlans)

When bridges are built AFTER KVM is installed, they show up in 'Virtual Network Interface' as Network source: Bridge virbr10: Host device vnet1 and Device model: virtio

It can look confusing when there are bridges built after the KVM install.

sharpy1064
  • 11
  • 3