Ping does not work on TAP interfaces with bridge

Question

I am trying to understand the linux bridging and local ip rules,

I have following topology on my linux laptop.

               br0 
     ___________|__________
     |                    |    
  |tap0                  tap1|
  |________Application_______|

The application above is creating 2 tap interfaces tap0 and tap1

I create the bridge and plug the tap interfaces to the bridge:

brctl addif br0 tap0
brctl addif br0 tap1

For ping to work, i need to add ip address to the interfaces, So I add 192.168.13.1 to tap0 and 192.168.13.2 to tap1

The application reads from one interface and writes to another interface, for both the interfaces.

Now if I run "ping 192.168.13.2 -I tap0"

PING 192.168.13.2 (192.168.13.2) from 192.168.13.1 tap0: 56(84) bytes of data.
From 192.168.13.1 icmp_seq=1 Destination Host Unreachable

tcpdump was showing arp could not be resolved, so I added static ARP entries:

arp -i tap0 -s 192.168.13.1 62:34:58:e7:8a:3a
arp -i tap1 -s 192.168.13.2 4a:6d:fa:51:7d:2d

brctl showmacs br0
port no mac addr        is local?   ageing timer
  2 4a:6d:fa:51:7d:2d   yes        0.00
  2 4a:6d:fa:51:7d:2d   yes        0.00
  1 62:34:58:e7:8a:3a   yes        0.00
  1 62:34:58:e7:8a:3a   yes        0.00

The bridge also seems to have learnt the MAC addresses.

However, the application and the tcpdump still receive 42 bytes packets showing ARP packets not resolved yet, and ping give Host unreachable mesg.

My current routing table is:

ip route ls table main
169.254.0.0/16 dev virbr0  scope link  metric 1000 linkdown 
192.168.13.0/24 dev tap1  proto kernel  scope link  src 192.168.13.2 
192.168.13.0/24 dev tap0  proto kernel  scope link  src 192.168.13.1 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1 linkdown

My current local routing table:

broadcast 192.168.13.0 dev tap1  proto kernel  scope link  src 192.168.13.2 
broadcast 192.168.13.0 dev tap0  proto kernel  scope link  src 192.168.13.1 
local 192.168.13.1 dev tap0  proto kernel  scope host  src 192.168.13.1 
local 192.168.13.2 dev tap1  proto kernel  scope host  src 192.168.13.2 
broadcast 192.168.13.255 dev tap1  proto kernel  scope link  src 192.168.13.2 
broadcast 192.168.13.255 dev tap0  proto kernel  scope link  src 192.168.13.1

I am having a feel that routing might not come into picture here: since this is a layer 2 broadcast domain. But since I am not well versed with linux bridging, I need some advice to proceed.

How can we make the ping work between tap0 and tap1.

Thanks Nayan

score 5 · Answer 1 · answered Dec 22 '19 at 22:10

I think you didn't grasp what's the role and usage of tun/tap interfaces, and that's the cause of your problems. Let me explain.

Consider the tap0 interface as the end of a virtual wire with two sides: the visible side: tap0, and the invisible side of the wire in the application: this invisible side has to be entirely handled by the application. The application will receive ethernet frames as data payload read on a file descriptor, if needed decode this data as ethernet (because it's in tap mode), then as ARP or IP and will then react to any needed event. In short your application requires a TCP/IP network stack, to be able to answer a ping.

Examples of such applications using tun/tap devices are openvpn or the combination of QEMU+VM+OS (QEMU is here to present this data payload to a network device driver of the VM's OS the same way this driver would see from an actual network device).

When you enslave tap0 and tap1 to the bridge, they become bridge ports: that's a hint they should not receive an IP (would it make sense to assign an IP to each port of your 24-ports switch?): those IPs are mostly ignored, except as an IP belonging to the host: a route with scope local is thus added to the local routing table. You could have added those IPs to the lo loopback interface with the same result. Beside this, there's no route that will work with those ~~interfaces~~ bridge ports, what matters now is their master br0.

What you should do is add an IP on the bridge itself (the route will then use the br0 interface) or instead you can create a veth pair, enslave one side to the bridge, and add an IP on the other side: the route will use the veth interface with the IP on it (and leave the bridge without IP which is cleaner imho).

Now for the remaining part it all depends on your application: it has to handle those 192.168.13.1 and 192.168.13.2 IPs itself, including first replying to ARP requests for them, before replying to ICMP echo requests.

If you're now discovering that the application's role is not just to create the tun/tap interfaces, then you should forget using an application and tun/tap interfaces: use network namespaces which duplicate as many network stacks as needed, create bridges where needed, and veth interfaces accross those namespaces. It can all be tested with ip netns and ip link add ... type veth ....

Here are some links to the first 3 parts of a tutorial making use of bridge and veth interfaces with network namespaces (alas they don't use ip netns which is easier to have the job done):

Fun with veth-devices, Linux bridges and VLANs in unnamed Linux network namespaces – I II III

Hi A.B thanks for an elaborate answer, You are correct, that I haven't grasped it to an expert level, I started from this blog: http://rrendec.mindbit.ro/post/tun-tap-tunnels/, here they are using TUN interfaces instead of TAP and without the application (TCP/IP stack simulating app) handling ICMP protocol. So if the ping can work on a TUN interface, why shouldn't it work with a TAP with some MAC layer configuration changes (eg: ARP's, may be having ip addresses in different subnets or /32) ? — Haswell, Dec 26 '19 at 08:55
https://serverfault.com/questions/974269/tun-tap-works-on-incoming-or-outgoing-packet I went through this post and now even the TUN interface blog seems a bit unclear. So a straight forward question is "If an application creates a TUN interface with 192.168.10.1/32, and I use ping to reach another IP address 192.168.10.2/32 on another TUN interface created by the same application, where will the packet from ping be delivered, to the application straight or first the kernel." — Haswell, Dec 26 '19 at 09:14
reading your tutorial: "The prototyping application receives/sends raw IP packets." . Now the "sample application" tun2.c is just an application handing everything on one side to the other side: the application cheats and is just a connector between the two tun interfaces. The tutorial alters local routes to still be able to send packets for local use "outside" (and it will loop back). But your question doesn't even use such an application. I never read an information about your applications. — A.B, Dec 26 '19 at 10:50
The answer to the straightforward question: if your system has ANY interface with 192.168.10.1/32 on it and you use ping to reach another IP address 192.168.0.2/32 on ANY OTHER interface on your system, the packet will entirely be handled by the kernel. Now you can replace "ANY" and "ANY OTHER" by whatever you like, the result will be the same. Unless you alter the local routing table as described in your tutorial. But what is your goal? — A.B, Dec 26 '19 at 10:58
Ok. So I realised this cannot be done. Closing this question — Haswell, Dec 30 '19 at 14:48

Ping does not work on TAP interfaces with bridge

1 Answers1

Linked