Mininet host doesn't reply to ping

Question

I am trying to implement MPLS on mininet and I was able to successfully do it. I am able to push, swap and pop tags correctly.

I am having difficulties when I am trying to ping from one host to another. This is the network that I am working with:

h1--s1--r1--r5--r8--r4--s4--h4

I noticed that when r4 delivers the packet to s4, s4 does nothing with it and it never reaches the host (h4), so I decided to remove the switches completely and just use the hosts and routers

h1--r1--r5--r8--r4--h4

and I started to capture what h4 is receiving, and I noticed he is receiving the ping from h1 (10.0.1.10) but h4 (10.0.4.10) never replies.

This is the output on h4:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on h4-eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
18:11:31.984633 IP 10.0.1.10 > 10.0.4.10: ICMP echo request, id 5767, seq 1, length 64
18:11:31.984961 ARP, Request who-has 10.0.4.1 tell 10.0.4.10, length 28
18:11:31.984970 ARP, Reply 10.0.4.1 is-at fe:65:4f:0c:2f:17 (oui Unknown), length 28
18:11:31.984972 IP 10.0.4.10.49609 > 172.16.219.2.domain: 52855+ PTR? 10.4.0.10.in-addr.arpa. (40)
18:11:31.984983 IP 10.0.4.1 > 10.0.4.10: ICMP net 172.16.219.2 unreachable, length 76
18:11:36.990650 IP 10.0.4.10.49609 > 172.16.219.2.domain: 52855+ PTR? 10.4.0.10.in-addr.arpa. (40)
18:11:36.990609 ARP, Request who-has 10.0.4.10 tell 10.0.4.1, length 28
18:11:52.006132 IP 10.0.4.10.38978 > 172.16.219.2.domain: 7045+ PTR? 1.4.0.10.in-addr.arpa. (39)
18:12:02.018454 IP 10.0.4.10.45969 > 172.16.219.2.domain: 62742+ PTR? 2.219.16.172.in-addr.arpa. (43)
18:12:02.018478 IP 10.0.4.1 > 10.0.4.10: ICMP net 172.16.219.2 unreachable, length 79

this is h4 routing table and it has a gateway:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         10.0.4.1        0.0.0.0         UG    0      0        0 h4-eth0
10.0.4.0        *               255.255.255.0   U     0      0        0 h4-eth0

any help is greatly appreciated.

UPDATE

netstat -s of h4 before testing

Ip:
    1329 total packets received
    0 forwarded
    0 incoming packets discarded
    1329 incoming packets delivered
    1329 requests sent out
Icmp:
    24 ICMP messages received
    0 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 8
        echo requests: 8
        echo replies: 8
    16 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        echo request: 8
        echo replies: 8
IcmpMsg:
        InType0: 8
        InType3: 8
        InType8: 8
        OutType0: 8
        OutType8: 8
Tcp:
    1 active connections openings
    1 passive connection openings
    0 failed connection attempts
    0 connection resets received
    2 connections established
    1305 segments received
    1305 segments send out
    0 segments retransmited
    0 bad segments received.
    0 resets sent
Udp:
    0 packets received
    0 packets to unknown port received.
    0 packet receive errors
    8 packets sent
UdpLite:
TcpExt:
    65 delayed acks sent
    856 packet headers predicted
    28 acknowledgments not containing data payload received
    805 predicted acknowledgments
    TCPRcvCoalesce: 14
    TCPOrigDataSent: 974
    TCPKeepAlive: 1
IpExt:
    InOctets: 3413476
    OutOctets: 3413252
    InNoECTPkts: 1329

netstat -s of h4 after testing

Ip:
    1384 total packets received
    0 forwarded
    0 incoming packets discarded
    1384 incoming packets delivered
    1384 requests sent out
Icmp:
    32 ICMP messages received
    0 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 16
        echo requests: 8
        echo replies: 8
    16 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        echo request: 8
        echo replies: 8
IcmpMsg:
        InType0: 8
        InType3: 16
        InType8: 8
        OutType0: 8
        OutType8: 8
Tcp:
    1 active connections openings
    1 passive connection openings
    0 failed connection attempts
    0 connection resets received
    2 connections established
    1352 segments received
    1352 segments send out
    0 segments retransmited
    0 bad segments received.
    0 resets sent
Udp:
    0 packets received
    0 packets to unknown port received.
    0 packet receive errors
    16 packets sent
UdpLite:
TcpExt:
    72 delayed acks sent
    872 packet headers predicted
    28 acknowledgments not containing data payload received
    828 predicted acknowledgments
    TCPRcvCoalesce: 16
    TCPOrigDataSent: 1000
    TCPKeepAlive: 1
IpExt:
    InOctets: 3421472
    OutOctets: 3421024
    InNoECTPkts: 1384

First I was using static routing, this are the routes:

r1 ip route add 10.0.4.0/24  via 172.16.1.2
r4 ip route add 10.0.1.0/24  via 172.16.4.2
r5 ip route add 10.0.1.0/24  via 172.16.1.1
r5 ip route add 10.0.4.0/24  via 172.16.8.2
r8 ip route add 10.0.1.0/24 via 172.16.8.1
r8 ip route add 10.0.4.0/24 via 172.16.4.1

Then I replace it with MPLS instead:

r1 ip route add 10.0.4.0/24 encap mpls 400 via inet 172.16.1.2
r5 ip -f mpls route add 400 as 400 via inet 172.16.8.2
r8 ip -f mpls route add 400 as 400 via inet 172.16.4.1
r4 ip -f mpls route add 400 dev r4-eth0

r4 ip route add 10.0.1.0/24 encap mpls 100 via inet 172.16.4.2
r8 ip -f mpls route add 100 as 100 via inet 172.16.8.1
r5 ip -f mpls route add 100 as 100 via inet 172.16.1.1
r1 ip -f mpls route add 100 dev r1-eth0  

any help is greatly appreciated.

UPDATE

A wireshark capture attached confirms that h4 is not replying

Answer 1

Listing 172.16.219.2 in /etc/resolv.conf doesn't appear to be helpful at present.

Test whether h4 can ping its own loopback address, then test pinging its 10.0.4.10 address. Take a peek to see if dmesg or syslog offer any hints.

Verify that iptables or other packet filter isn't making a policy decision that prevents delivery of the inbound packet (tcpdump saw it but perhaps the kernel endpoint does not) or prevents delivery of an outbound reply.

View the first few sections of h4's netstat -s output, and note which counters increment between start and end of your experimental probing.

You showed us a single inbound packet at 18:11:31.984633, followed by ARP cache miss for default router. If h1 is continuously sending pings, and h4 doesn't have a cache miss, do we see different results?

H1 is stimulating the system-under-test with an ICMP packet. While using tcpdump to listen for three-way handshake, try another stimulus from h1, perhaps port 22 or port 80: telnet 10.0.4.10 80

EDIT: You observe that "[after removing] the routes to use MPLS the problem began." This suggests packets are not leaving via default route. Try an outbound stimulus (perhaps ping) to help identify whether r4 even sees packets that you hope are egressing from h4.

And on netstat -s, the +8 ICMP dest unreach seems like trouble, again pointing at static route vs your default route. It matches with +0 ICMPs sent, something you'd already confirmed with tcpdump.