debian lenny xen bridge networking problem

Question

DomU isn't talking to the world, but it talks to Dom0.

Here are the tests that I made:

Dom0 (external networking is working):

ping 188.40.96.238 #Which is Domu's ip

PING 188.40.96.238 (188.40.96.238) 56(84) bytes of data.
64 bytes from 188.40.96.238: icmp_seq=1 ttl=64 time=0.092 ms

DomU:

ping 188.40.96.215 #Which is Dom0's ip

PING 188.40.96.215 (188.40.96.215) 56(84) bytes of data.
64 bytes from 188.40.96.215: icmp_seq=1 ttl=64 time=0.045 ms

ping 188.40.96.193 #Which is the gateway - fail
PING 188.40.96.193 (188.40.96.193) 56(84) bytes of data.
^C
--- 188.40.96.193 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1013ms

The system is debian lenny with a normal setup.

Here is my configs:

uname -a

Linux green0 2.6.26-2-xen-686 #1 SMP Wed Aug 19 08:47:57 UTC 2009 i686 GNU/Linux

cat /etc/xen/green1.cfg |grep -v '#'

kernel      = '/boot/vmlinuz-2.6.26-2-xen-686'
ramdisk     = '/boot/initrd.img-2.6.26-2-xen-686'
memory      = '2000'

root        = '/dev/xvda2 ro'
disk        = [
                  'file:/home/xen/domains/green1/swap.img,xvda1,w',
                  'file:/home/xen/domains/green1/disk.img,xvda2,w',
              ]


name        = 'green1'

vif         = [ 'ip=188.40.96.238,mac=00:16:3E:1F:C4:CC' ]

on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'

ifconfig

eth0      Link encap:Ethernet  HWaddr 00:24:21:ef:2f:86  
          inet addr:188.40.96.215  Bcast:188.40.96.255  Mask:255.255.255.192
          inet6 addr: fe80::224:21ff:feef:2f86/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3296 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2204 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:262717 (256.5 KiB)  TX bytes:330465 (322.7 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

peth0     Link encap:Ethernet  HWaddr 00:24:21:ef:2f:86  
          inet6 addr: fe80::224:21ff:feef:2f86/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:3407 errors:0 dropped:657431448 overruns:0 frame:0
          TX packets:2291 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:319941 (312.4 KiB)  TX bytes:338423 (330.4 KiB)
          Interrupt:16 Base address:0x8000 

vif2.0    Link encap:Ethernet  HWaddr fe:ff:ff:ff:ff:ff  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:27 errors:0 dropped:0 overruns:0 frame:0
          TX packets:151 errors:0 dropped:33 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:1164 (1.1 KiB)  TX bytes:20974 (20.4 KiB)

ip a s

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: peth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 00:24:21:ef:2f:86 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::224:21ff:feef:2f86/64 scope link 
       valid_lft forever preferred_lft forever
4: vif0.0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
5: veth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
6: vif0.1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
7: veth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
8: vif0.2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
9: veth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
10: vif0.3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
11: veth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN 
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
12: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 00:24:21:ef:2f:86 brd ff:ff:ff:ff:ff:ff
    inet 188.40.96.215/26 brd 188.40.96.255 scope global eth0
    inet6 fe80::224:21ff:feef:2f86/64 scope link 
       valid_lft forever preferred_lft forever
14: vif2.0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 32
    link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fcff:ffff:feff:ffff/64 scope link 
       valid_lft forever preferred_lft forever

brctl show

bridge name bridge id       STP enabled interfaces
eth0        8000.002421ef2f86   no      peth0
                            vif2.0

ip r l

Dom0:

188.40.96.192/26 dev eth0  proto kernel  scope link  src 188.40.96.215
default via 188.40.96.193 dev eth0

DomU:

188.40.96.192/26 dev eth0  proto kernel  scope link  src 188.40.96.238
default via 188.40.96.193 dev eth0

Post a netstat -nr so we can see the routes on the machine as well, Please. — Scott Lundberg, Sep 29 '09 at 22:19

score 2 · Answer 1 · answered Mar 07 '10 at 15:14

the default bridge script does a lot of weird things to make the eth0/peth0 devices.. I have much better luck setting it up in /etc/network/interfaces as

# The primary network interface
auto xen-br0
iface xen-br0 inet static
        address 10.2.2.44
        gateway 10.2.2.1
        netmask 255.255.255.0
        bridge_ports eth0
        bridge_stp off
        bridge_fd 0

and then in xend-config.sxp:

(vif-script vif-bridge bridge=xen-br0)

that way debian sets up the bridge and xen leaves it alone.

Do you have access to the routers and switches? Can you run, or have someone else run:

show ip arp 188.40.96.238
show mac-address-table address 0016.3E1F.C4CC

(or whatever commands are appropriate for the devices you have) That would confirm if your domU is even visible to the rest of the network.

score 1 · Answer 2 · answered Oct 28 '10 at 17:09

Make sure the switch the dom0 machine is connected to allow multiple MAC addresses per port. I had configured a cisco switch with each port set to "Desktop". This enabled PortFast but also disabled multiple MAC addresses on the port. All packets from the domU VM were silently dropped. Switching the port setting back to "None" inside the switch resolved this problem.

score 0 · Answer 3 · answered Sep 29 '09 at 22:24

0

When the DomU can talk to the dom0 but not the outside world, while the dom0 can over the same bridge, then it's most likely that you've got a firewall on the dom0 that is catching the domU packets.

Given that I treat the bridge as a switch "on the network", and not something inside the sphere of control of the dom0, I just turn off the dom0's practice of running packets from the bridge through the dom0's firewall:

sysctl {
    "net.bridge.bridge-nf-call-arptables": value => "0";
    "net.bridge.bridge-nf-call-iptables": value => "0";
    "net.bridge.bridge-nf-call-ip6tables": value => "0";
    "net.bridge.bridge-nf-filter-vlan-tagged": value => "0";
}

answered Sep 29 '09 at 22:24

womble

96,255
29
175
230

I've tried the above by putting the above options in sysctl.conf also here is iptables-save: # Generated by iptables-save v1.4.2 on Wed Sep 30 08:05:42 2009 *filter :INPUT ACCEPT [4286:351627] :FORWARD ACCEPT [779:58367] :OUTPUT ACCEPT [3638:532622] COMMIT # Completed on Wed Sep 30 08:05:42 2009 – Alexandru Plugaru Sep 30 '09 at 06:06
I did execute sysctl -p /etc/sysctl.conf but the problem persists – Alexandru Plugaru Sep 30 '09 at 06:09

debian lenny xen bridge networking problem

3 Answers3