2

The situation is a VM running under kvm/libvirt/qemu (10.202.7.128) pinging another hardware-based system (10.202.7.4) via the hypervisor's bridge. Everything works OK but 'tcpdump -neli any icmp and host 10.202.7.128' shows three entries per request:

14:42:39.921785 P 52:54:00:14:71:1b 
ethertype IPv4 (0x0800), length 100: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 2485, seq 1, length 64

The second entry is identical except the P is replaced with "Out", the third entry is identical to the first. This is followed by three replies from the same MAC address for 10.202.7.4 with the only difference being the P, Out, P sequence.

Why are there three entries per ping and what does the P in the direction mean? I've searched the web in vain unless I searched for the wrong thing.

Update:

Based on Zoredache's answer I clanged the interface specification to the three involved: vnet5 (the guest VM's NIC on the hypervisor), br0 (the hypervisor's bridge) and eno1 (the physical NIC identified as the bridge port for the bridge) and got one request/reply for each ping when testing each interface so I'm (wildly) guessing that's the reason for the three entries using "any". I'm still baffled about the P and sequence of P, Out, P.
Any insight?

Further update (and I want to thank Anton for his research, much appreciated.)

Ran test again with four tcpdump specifications, all '-neli <interface> icmp and host 10.202.7.128' with interfaces vnet5, br0, eno1 and "any". Noted timestamps to determine what "any" entry matched the individual interface entries.

"Any" results (vnet5/10.202.7.128 is 52:54:00:a2:bd:48, 10.202.7.4 is b0:00:b4:02:93:6e)

00:06:57.775050   P 52:54:00:a2:bd:48 ethertype IPv4 (0x0800), length 100: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775074 Out 52:54:00:a2:bd:48 ethertype IPv4 (0x0800), length 100: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775050   P 52:54:00:a2:bd:48 ethertype IPv4 (0x0800), length 100: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775940   P b0:00:b4:02:93:6e ethertype IPv4 (0x0800), length 100: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64
00:06:57.775956 Out b0:00:b4:02:93:6e ethertype IPv4 (0x0800), length 100: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64
00:06:57.775940   P b0:00:b4:02:93:6e ethertype IPv4 (0x0800), length 100: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64

vnet5 results

00:06:57.775050 52:54:00:a2:bd:48 > b0:00:b4:02:93:6e, ethertype IPv4 (0x0800), length 98: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775956 b0:00:b4:02:93:6e > 52:54:00:a2:bd:48, ethertype IPv4 (0x0800), length 98: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64

br0 results

00:06:57.775050 52:54:00:a2:bd:48 > b0:00:b4:02:93:6e, ethertype IPv4 (0x0800), length 98: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775940 b0:00:b4:02:93:6e > 52:54:00:a2:bd:48, ethertype IPv4 (0x0800), length 98: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64

eno1 results

00:06:57.775074 52:54:00:a2:bd:48 > b0:00:b4:02:93:6e, ethertype IPv4 (0x0800), length 98: 10.202.7.128 > 10.202.7.4: ICMP echo request, id 27288, seq 1, length 64
00:06:57.775940 b0:00:b4:02:93:6e > 52:54:00:a2:bd:48, ethertype IPv4 (0x0800), length 98: 10.202.7.4 > 10.202.7.128: ICMP echo reply, id 27288, seq 1, length 64

The 775050 timestamp corresponds to vnet5 and br0, the expected first "hop" (vnet5 to br0). The 775074 corresponds to eno1 which is the expected second "hop" (br0 to eno1). I have no explanation for the repeat of the second 775050 entry because it is still a request.

The 775940 timestamp corresponds to eno1 and br0, the expected first "hop" of the reply (eno1 to br0). The 775956 timestamp corresponds to vnet5, the expected second "hop" of the reply (br0 to vnet5). Again, I have no explanation for the repeat of the second 775940 entry because it is still a reply.

Final note, both "Out" entries refer to br0 transmitting to an interface (eno1 on outbound, vnet5 on inbound).

If anyone has an explanation for the repeat of the two entries please respond.

Community
  • 1
Senior Geek
  • 55
  • 9
  • Are you running `tcpdump` on the hypervisor or inside the guest? Also, is the second guest running on the same hypervisor? Are the two guests and/or the hypervisor in the same subnet? – Joel C Jun 25 '19 at 20:56
  • tcpdump is being run on the hypervisor while the guest is pinging, the source, destination and hypervisor are on the same subnet. – Senior Geek Jun 27 '19 at 17:02

2 Answers2

3

I've looked in the source code of the tcpdump.

Meanings of the packet type field:

  • In - packet addressed to host
  • Out - outgoing packet
  • B - broadcast
  • M - multicast
  • P - packet addressed to other host

Update

Duplicated packets in the tcpdump output caused by -i any option.

When a ethernet frame passed through linux bridge and you use -i any option of the tcpdump, you will see this frame several times. First time you see the frame on the input interface (vnet5), second time on the bridge interface itself (br0), and third time on the output interface (eno1). Seeing transit frames on the bridge interface is the side effect of implementation of ethernet bridging in the linux kernel.

Also, when a bridge interface have several ports (slave interfaces), in some cases you see copies of frames on every port, when an ethernet frame is flooded. It happens for broadcast, multicast and unknown unicast frames.

Community
  • 1
Anton Danilov
  • 5,082
  • 2
  • 13
  • 23
1

Most of your duplication is caused by the -i any option you have passed.

When you do that you tcpdump will capture on every interface, and in the case of a router, or a bridge or something where a packet is being forwarded between interfaces it will return a copy of the packet for every interface that it hits.

I am not very familiar with KVM, but I believe if you have 2 VMs connected by a bridge interface you would have on virtual interface per VM, and the bridge interface.

Anyway, if you don't want duplication, restrict tcpdump to a single interface, by providing the name of a specific interface to the -i option. Though for some problems you may actually want all interfaces, and in that case you just have to understand that you are going to see duplication.

Zoredache
  • 130,897
  • 41
  • 276
  • 420