LVS-TUN: ifconfig shows errors and collisions for tunl0 interface?

Question

Check_MK has sent me an email as follows:

***** Nagios *****

Notification Type: PROBLEM

Service: Interface 5
Host: foo
Address: x.y.z.t
State: CRITICAL

Date/Time: Fri May 3 10:02:40 ICT 2013

Additional Info: CRIT - [tunl0] (up) speed unknown, in: 3.39MB/s, out: 0.00B/s, out-errors: 100.00%(!!) = 0.1

Running ifconfig, I got:

tunl0     Link encap:IPIP Tunnel  HWaddr   
          inet addr:x.y.z.t  Mask:255.255.255.255
          UP RUNNING NOARP  MTU:1480  Metric:1
          RX packets:92101704629 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:652 dropped:0 overruns:0 carrier:0
          collisions:652 txqueuelen:0 
          RX bytes:18941091817671 (17.2 TiB)  TX bytes:0 (0.0 b)

Pay attention to the errors and collisions. I know that a nonzero value of the collisions field indicates the possibility of network congestion. But:

What may be the exact cause? How can I troubleshoot?
Is there any similar to ethtool for IPIP Tunnel interface?

modinfo ipip

filename:       /lib/modules/2.6.18-194.17.1.el5/kernel/net/ipv4/ipip.ko
license:        GPL
srcversion:     288C625C7521D577F7AD9E4
depends:        tunnel4
vermagic:       2.6.18-194.17.1.el5 SMP mod_unload gcc-4.1
module_sig: 883f3504ca37590565662cff69dd0be11277ff0a08d3a3...

ip tunnel show

tunl0: ip/ip  remote any  local any  ttl inherit  nopmtudisc

UPDATE at Mon May 6 10:05:01 ICT 2013

@Danila Ladner: Searching through Google, I found this link has same opinion with you:

My tunnel does not work:

ifconfig tunl<n> reports errors and collisions

Did you use ifconfig, perhaps ifconfig ... pointopoint ... to set up your tunnel?

Shut it down; delete it; start again with ip.

But could you please elaborate further?

@Sergey Vlasov:

tunl0     Link encap:IPIP Tunnel  HWaddr   
          inet addr:x.y.z.t  Mask:255.255.255.255
          UP RUNNING NOARP  MTU:1480  Metric:1
          RX packets:81621711099 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:692 dropped:0 overruns:0 carrier:0
          collisions:692 txqueuelen:0 
          RX bytes:16915649263419 (15.3 TiB)  TX bytes:120 (120.0 b)

I don't understand why there are 2 transmitted packets from tunl0 interface? I'm going to setup an event handler to run tcpdump whenever collisions counter is increased. Let's wait to see what happens.

UPDATE at Tue May 7 14:05:39 ICT 2013

@Danila Ladner: To exclude the possibility, I have tried your suggestion:

ifdown tun0
modprobe -r ipip
modprobe ipip
ip addr add dev tunl0 x.y.z.t/32 brd x.y.z.t
ip link set tunl0 up

I'm waiting to see if the problem is resolved:

tunl0     Link encap:IPIP Tunnel  HWaddr   
          inet addr:x.y.z.t  Mask:255.255.255.255
          UP RUNNING NOARP  MTU:1480  Metric:1
          RX packets:19630041 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:4083271398 (3.8 GiB)  TX bytes:0 (0.0 b)

if ifconfig then shut it down; delete it; start again with ip. — Danila Ladner, May 03 '13 at 06:05
When did it start doing collisions? Did anyone took tunnel down with ifconfig and then tried to bting it up with ip? — Danila Ladner, May 03 '13 at 06:12
_if ifconfig then shut it down; delete it; start again with ip._ --> Could you please explain why? _When did it start doing collisions?_ --> a random time. It has increased to `collisions:732`. _Did anyone took tunnel down with ifconfig and then tried to bring it up with ip?_ --> No, I'm pretty sure. — quanta, May 03 '13 at 06:15
@DanilaLadner: feel free to post your comment as an answer. Sure, I'll upvote and accept + bounty (if it solved my problem). — quanta, May 07 '13 at 07:23

Sergey Vlasov · Answer 1 · 2013-05-05T20:14:21.277

5

The collisions counter for an ipip tunnel interface is increased in two cases:

If the next hop of the encapsulated packet is the same tunnel interface: ipip.c line 437.
If the path MTU of the next hop for the encapsulated packet is less than 68: ipip.c line 447.

Both of these cases can usually happen only if the encapsulated traffic loops back into the same tunnel (the first case is a direct looping, the second case happens when the path MTU is reduced down to zero due to some more complicated looping which was not immediately detected by the first condition). One possible cause is that the normal route for encapsulated packets was temporarily down, and the next best route for these packets happened to be the tunnel itself, resulting in a loop.

However, in the LVS-TUN case nothing should have been sent to the tunnel at all (the tunnel interface in this case is receive-only), unless some misguided software added unneeded routes through tunl0.

edited May 05 '13 at 20:14

answered May 05 '13 at 19:55

Sergey Vlasov

6,288
1
21
30

_However, in the LVS-TUN case nothing should have been sent **to** the tunnel at all_ --> It should be "from". – quanta May 06 '13 at 07:34
The value of `collisions` has increased to 695 and the event handler was activated. Now I have the `.pcap` files on `tunl0` (receving) and the `eth0` (sending) at that time. What or where should I pay attention to? – quanta May 06 '13 at 07:43
I suspect that you will not see anything useful in `.pcap` at all, unless you managed to capture some packets which were transmitted to the `tunl0` interface (but were dropped and counted in `collisions` instead of being encapsulated and sent through some other interface). Look for packets on `tunl0` which have something else than your VIP as the destination. – Sergey Vlasov May 06 '13 at 07:54
_It should be "from"_ → Actually it should be “to”, because the problem is caused by packets which were attempted to be transmitted through the `tunl0` interface (in a normal tunnel such packets would then get the tunnel header prepended, then they will be reinjected into the IPv4 stack to be sent to the remote tunnel endpoint). In the “from” direction you actually receive all of the incoming traffic for your VIP. – Sergey Vlasov May 06 '13 at 08:03

score 1 · Accepted Answer · edited May 07 '13 at 12:42

As quanta noted I suggested him to take the tunnel down if it was built with ifconfig and rebuild it with ip. As I had a similar issue on Centos 5 kernel 2.6.25 a few years back, In my case it resolved the issue, But I was also consulting net guys and devs in IRC why that was an issue as I needed that route on production box and needed to schedule a downtime to nuke it. I do not remember exactly and as of now do not have any hard proof but Kuznetsov (original big contributor to the kernel source on the matter suggested to rebuild it with ip as he has seen issues with ifconfig. I hope this helps quanta to resolve his issue.

OFF TOPIC: So, the bottom line is i am quite dumb myself using a lot of ifconfig and it is hard to switch to ip, as long as I continue dealing with old Solaris 8 boxes and bsd boxes.

LVS-TUN: ifconfig shows errors and collisions for tunl0 interface?

2 Answers2