6

I have 5 physical machines running Ubuntu 14.04, all of them are together in a network. These all have static IP addresses. Each of these machines has 1 to 4 virtual machines via KVM/QEMU running Ubuntu 14.04. I'm allowed to use DHCP to get a non-static address for all the virtual machines on the physical network. But for my distributed software, I need all the virtual machines to have static IP addresses within the virtual network (as I cannot get more on the physical network). This virtual network should however span all the guests on all the virtual machines. Also, one of the virtual machines should also have a static address on the physical network.

I have managed to create a virtual network in a single host for all the virtual machines, so these can have static IPs there, but if I do this per host, it won't work. I have also created a bridged interface for the static IP on the physical network on one of the virtual machines.

This is my desired scheme:

123.45.144.0/28 (physical network)
192.168.0.0/16 (virtual network for all the vms)

123.45.147.3 (host)
    192.168.122.101, 123.45.147.49 (vm)
    192.168.122.102 (vm)
    192.168.122.103 (vm)
    192.168.122.104 (vm)

123.45.147.4 (host)
    192.168.122.105 (vm)

123.45.147.5 (host)
    192.168.122.106 (vm)

123.45.147.6 (host)
    192.168.122.107 (vm)
    192.168.122.108 (vm)

123.45.147.7 (host)
    192.168.122.109 (vm)
    192.168.122.110 (vm)

Here are outputs of ifconfig:

One of the hosts:

eth0      Link encap:Ethernet  HWaddr 5c:f9:dd:75:07:72  
          inet addr:123.45.147.3  Bcast:123.45.151.255  Mask:255.255.248.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

virbr0    Link encap:Ethernet  HWaddr 52:54:00:bd:2b:39  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

Virtual machine that has both bridged interface (eth1) and an interface for the virtual network (eth0)

eth0      Link encap:Ethernet  HWaddr 52:54:00:59:2d:c8  
          inet addr:192.168.122.101  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

eth1      Link encap:Ethernet  HWaddr 52:54:00:4d:d0:59  
          inet addr:123.45.147.49  Bcast:123.45.151.255  Mask:255.255.248.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

After hours of googling, I was unable to find a similar setup of the problem I'm facing. As my familiarity with networks is rather low, I'd be appreciative for any leads to a solution.

Eddie C.
  • 535
  • 1
  • 3
  • 12
Sha
  • 61
  • 1
  • 2
  • 1
    if you need connect your virtual network, you can try to use openvswitch for layer 2 switch. – c4f4t0r Jun 27 '15 at 11:01
  • When you create the virtual network on each host, notice ``ifconfig`` shows "virbr0" having address "192.168.122.1". Try editing the network config file on each host after creating the virtual network so that "virbr0" contains a different ip (e.g. 192.168.122.2 on host 2, 192.168.122.3 on host 3, 192.168.122.4 on host 4). You are probably experiencing duplicate ip issues because it is easy to forget that your host is a member of that virtual network through virbr0. You have to change the ip address in the config files so that the settings will survive a reboot. – Gordster Mar 28 '20 at 23:44

2 Answers2

0

When you create the virtual network on each host, notice ifconfig shows "virbr0" as having address "192.168.122.1". Try editing the virtual network config file on each host after creating the virtual network so that "virbr0" contains a different ip address (e.g. 192.168.122.2 on host 2, 192.168.122.3 on host 3, 192.168.122.4 on host 4, 192.168.122.5 on host 5).

You are probably experiencing duplicate ip issues because it is easy to forget that your host is a member of that virtual network through virbr0. You have to change the ip address in the config files so that the settings will survive a reboot.

You may have to enable packet forwarding to make this work

sudo sysctl net.ipv4.ip_forward=1

Gordster
  • 174
  • 8
  • I would prefer to do 192.168.122.1 and 192.168.123.1, in this way, every host has a dedicated virtual subnet – c4f4t0r Mar 16 '23 at 09:25
0

If you have a separate ethX interface on the system with the connection to the physical network, create a second bridge on that interface. You can then configure the physical address on that virtual bridge.

The static address can be configured on the virtual machine. This makes keeping a stable address space simpler.

BillThor
  • 27,737
  • 3
  • 37
  • 69
  • I only have a separate eth1 interface on one of the machines. From what you said, I understand I would create another physical network. It is not possible to create another physical network spanning all the hosts. Would it be somehow possible to create a subnetwork with the first host as a router and then somehow assign IPs form the physical network to both the other hosts and the vms? – Sha Jun 29 '15 at 04:10
  • @Sha You a separate network, or route them over the existing routing structure. You can add alias addresses with the Internet addresses, which would have the default route. (Local traffic will bind to the smaller 192.168.* address.) To access local addresses, use whatever naming scheme you choose: a separate domain, service based domains for the internet (www.example.com, mail.example.com), or alternate names. Firewall configuration is trickier when you have multiple addresses with different security requirements on the same port. – BillThor Jun 30 '15 at 04:24