5

I am trying to create two VMs which are both connected to the same private network. I'm using Linux with qemu-kvm 1.0.

My plan of attack has been this:

brctl addbr bridge
ifconfig bridge up
tunctl -t tap1
tunctl -t tap2
ifconfig tap1 up
ifconfig tap2 up
brctl addif bridge tap1
brctl addif bridge tap2
qemu-kvm -net nic,macaddr=52:54:00:11:22:33 -net tap,ifname=tap1 disk1.img
qemu-kvm -net nic,macaddr=52:54:00:44:55:66 -net tap,ifname=tap2 disk2.img

Once booted, I give the first machine the IP address 192.168.100.5, and the second 192.168.100.10.

At this point, when I try pinging one VM from the other, there is no ping response. However, using Wireshark, I see that ARP requests are sent and responded to, and I verified that the ARP caches do contain the information on the other VMs. Yet no ping replies are generated (as seen via Wireshark).

Next, I tried giving the bridge an IP address of 192.168.100.1. After doing this, pinging between VMs works, but there is still a problem: now all requests appear to be coming from the bridge itself. For example, if I connect from one VM to the other's FTP server, running netstat on the VM with the FTP server shows that 192.168.100.1 is the source. Connections work as they do across NAT, but as with NAT, the source address is not that of the originating machine. I've tried this with net.ipv4.ip_forward both on and off, and masquerading (iptables -t nat -A POSTROUTING -j MASQUERADE) on and off, with the same results.

What I really want is for my VMs to act as though they're plugged into a switch: it should be transparent. I'm more concerned with the source address looking like the bridge than the bridge requiring an IP. The latter is somewhat annoying, but the former is a blocker for me.

Chris
  • 151
  • 1
  • 3
  • It turns out that this is an issue on Debian, but not on Arch (I've yet to test other distributions). I have not tracked down _why_ this happens on Debian, but it does allow a workaround for the time being. – Chris Jun 29 '12 at 16:13
  • I finally tracked down a solution. The following sysctl variables have to be set: `net.bridge.bridge-nf-call-arptables = 0, net.bridge.bridge-nf-call-ip6tables = 0, net.bridge.bridge-nf-call-iptables = 0` Some explanation can be found [here](https://bugzilla.redhat.com/show_bug.cgi?id=512206). – Chris Jul 03 '12 at 21:07

3 Answers3

2

I've seen iptables interfering with bridge traffic before (even though it shouldn't AFAIK). You certainly don't want any NAT related rules, but I think that the FORWARD chain needs to accept the packets. I would suggest testing this with no iptables rules and a default ACCEPT policy on the FORWARD chain.

A couple other things to check:

  • Does brctl show verify that tap1 and tap2 are in bridge?
  • Does brctl showmacs bridge show the MAC addresses for the two VMs?
mgorven
  • 30,615
  • 7
  • 79
  • 122
  • I verified that I've got a default accept policy on FORWARD, so unfortunately that's not the issue. The tap devices are attached, and the VMs are shown with showmacs (but so are the tap devices, if that matters). I've also had a chance to try something similar to this on a machine with Open vSwitch, and I got the desired outcome (source IP looks right), so perhaps I'll have to switch to that. – Chris Apr 26 '12 at 15:13
1

Set your tap interfaces to promiscuous mode.

ifconfig tap1 promisc up
ifconfig tap2 promisc up
jscott
  • 24,484
  • 8
  • 79
  • 100
loloudoudara
  • 11
  • 1
  • Thanks, but unfortunately, there was no change after this: communication was still down and the IP addresses were still mangled. – Chris Jun 29 '12 at 16:14
1

ARP requires routing to work also.

A common problem is that you have both an IP address assigned to both interfaces in the same subnet, on the host system's kernel. If you do this ARP replies will break - b/c only one interface will get replies. Make sure that you have a clean, single route back to the subnet.

In the example above, if the tap1 and tap2 interfaces both had IP address in the Linux host's kernel - in the same subnet (192.168.100.0/24) ARP replies would break. If the VM's did not require connectivity to the host, then neither would need an IP in the host kernel.

EdH
  • 131
  • 2