5

I have been researching this problem for a few days and have not found an answer yet. Your help will be really appreciated!

I have a few VMs (Virtual Machines) running on a physical server. The server uses Linux bridge (br100) to connect these VMs together:

# brctl show
bridge name     bridge id               STP enabled     interfaces
br100           8000.984be15fe7e3       no              eth1.1729
                                                        vnet0
                                                        vnet1

vnet0 and vnet1 are VMs' virtual NICs.

br100 (physical server) is assigned to IP 172.16.0.11. The VM connected to vnet1 is assigned to 172.16.0.3, the one to vnet0 is 172.16.0.5.

So far so good. 172.16.0.3 can ping 172.16.0.5 with no problems.

Now I'm trying to set up 172.16.0.3 as an router (openvpn server if that matters) to subnet 10.8.0.0/16.

Here comes my problem: machines in 10.8.0.0/16 (in this case 10.8.0.6) can ping 172.16.0.3, but is unable to ping 172.16.0.5.

(I think) I have ruled out all obvious causes: ip_forward turned on, iptables flushed, etc. Now I have narrowed the cause to: br100 is not forwarding packets as it should!

When I ping 172.16.0.5 from 10.8.0.6, packets were delivered to vnet1 (VM 172.16.0.3) on the physical server:

# tcpdump -leni vnet1 icmp
tcpdump: WARNING: vnet1: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet1, link-type EN10MB (Ethernet), capture size 65535 bytes
07:45:03.858356 02:16:3e:6a:42:57 > 02:16:3e:02:40:82, ethertype IPv4 (0x0800), length 98: 
10.8.0.6 > 172.16.0.5: ICMP echo request, id 63242, seq 1046, length 64
07:45:04.858239 02:16:3e:6a:42:57 > 02:16:3e:02:40:82, ethertype IPv4 (0x0800), length 98:
10.8.0.6 > 172.16.0.5: ICMP echo request, id 63242, seq 1047, length 64
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel

But are not forwarded to vnet0 (172.16.0.5): 

# tcpdump -leni vnet0 icmp
tcpdump: WARNING: vnet0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet0, link-type EN10MB (Ethernet), capture size 65535 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

I also followed the advices in this post and set 0 to /proc/sys/net/bridge/bridge-nf-*, but it didn't seem to help.

In addition to flushing iptables filters, I also turned on TRACE in iptables, which showed that these packets never hit iptables.

Any other reasons why Linux bridge would not forward packets between ports?

Kenneth
  • 199
  • 1
  • 2
  • 6
  • Do the hosts on 10.8.0.0/16 know how to reach 172.16.0.5? `route -n` on one of those... – S19N Jan 07 '12 at 16:22
  • eth1.1729 is in VLAN 1729 physically attached to a network? What IP and MAC-Adress are set for that interface? – Nils Jan 07 '12 at 20:50
  • check if your bridge address tables are populated correctly by issuing `brctl showmacs br100`. Also, check if the MAC addresses match - 02:16:3e:6a:42:57 should belong to 172.16.0.3, 02:16:3e:02:40:82 should belong to 172.16.0.5. – the-wabbit Jan 08 '12 at 00:36
  • @S19N, I configured 172.16.0.5 to route via 172.16.0.3: 10.8.0.0 172.16.0.3 255.255.255.0 UG 0 0 0 eth0 – Kenneth Jan 08 '12 at 13:39
  • @syneticon-dj, `# brctl showmacs br100 port no mac addr is local? ageing timer 2 02:16:3e:02:40:82 no 10.96 4 02:16:3e:25:20:62 no 3.16 3 02:16:3e:6a:42:57 no 4.92 1 98:4b:e1:5f:e7:e3 yes 0.00 2 fe:16:3e:02:40:82 yes 0.00 4 fe:16:3e:25:20:62 yes 0.00 3 fe:16:3e:6a:42:57 yes 0.00` – Kenneth Jan 08 '12 at 13:42
  • @syneticon-dj, `# arp -a ? (172.16.0.10) at 02:16:3e:25:20:62 [ether] on br100 ? (172.16.0.5) at 02:16:3e:02:40:82 [ether] on br100 ? (172.16.0.3) at 02:16:3e:6a:42:57 [ether] on br100` They all look right to me – Kenneth Jan 08 '12 at 13:48
  • @Nils, eth1.1729 is attached to network. Its mac is 98:4b:e1:5f:e7:e3 , and it doesn't not have an IP (IP 172.16.0.11 is assigned to br100) – Kenneth Jan 08 '12 at 13:57

1 Answers1

4

Found the answer! Linux ethernet bridge consults ebtables to decide which packets to forward and which to drop. Flushing ebtables solved my problem.

Kenneth
  • 199
  • 1
  • 2
  • 6
  • Sweet! Always fun to answer your own question. Don't forget to come back and accept it when the time limit is up. =) – Wesley Jan 10 '12 at 02:47