1

I'm currently creating a new container on a fresh server and I'm facing a network issue. Basically the container cannot reach anything on the Internet i.e. every request timeout.

Here are the information related to the host and the lxc configuration:

  • uname -r output on the server: 4.9.185-xxxx-std-ipv6-64.
  • lxc-ls --version: 3.0.3
  • cat /proc/sys/net/ipv4/ip_forward: 1
  • cat /etc/lxc/lxc.conf:
lxc.lxcpath = /home/lxc
  • cat /etc/default/lxc-net: 
USE_LXC_BRIDGE="true"
LXC_BRIDGE="lxc-bridge-nat"
LXC_ADDR="192.168.100.1"
LXC_NETMASK="255.255.255.0"
LXC_NETWORK="192.168.100.0/24"
LXC_DHCP_RANGE="192.168.100.2,192.168.100.254"
LXC_DHCP_MAX="253"
LXC_DHCP_CONFILE=""
LXC_DOMAIN="lxc"

LXC_IPV6_ADDR="fc27::216:3eff:fe00:1"
LXC_IPV6_MASK="64"
LXC_IPV6_NETWORK="fc27::/64"
LXC_IPV6_NAT="true"
  • cat /home/lxc/test/config
# Template used to create this container: /usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -d debian -r buster -a amd64
# Template script checksum (SHA-1): 273c51343604eb85f7e294c8da0a5eb769d648f3
# For additional config options, please look at lxc.container.conf(5)

# Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
# (Be aware this has security implications)

# Auto start configuration
# Mount shared folder so the container can access letsencrypt certificates
# into its own /share folder

# Distribution configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.arch = linux64

# Container specific configuration
lxc.start.auto = 1
lxc.rootfs.path = dir:/home/lxc/test/rootfs
lxc.uts.name = test

# Network configuration
lxc.net.0.type  = veth
lxc.net.0.flags = up
lxc.net.0.link  = lxc-bridge-nat

With this setup I can successfully ping the container or the host from the container through the lxc-bridge-nat interface. But inside the container the command ping 8.8.8.8 does not give any results:

PING 8.8.8.8 (8.8.8.8): 56 data bytes
^C--- 8.8.8.8 ping statistics ---
18 packets transmitted, 0 packets received, 100% packet loss

Here is the iptables configuration on the host:

 Generated by xtables-save v1.8.3 on Wed Jul 17 11:10:37 2019
*filter
:INPUT ACCEPT [245428:1063335794]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [228241:18791314]
:f2b-sshd - [0:0]
:f2b-recidive - [0:0]
-A INPUT -i lxc-bridge-nat -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -i lxc-bridge-nat -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i lxc-bridge-nat -p tcp -m tcp --dport 67 -j ACCEPT
-A INPUT -i lxc-bridge-nat -p udp -m udp --dport 67 -j ACCEPT
-A INPUT -p tcp -j f2b-recidive
-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
-A FORWARD -o lxc-bridge-nat -j ACCEPT
-A FORWARD -i lxc-bridge-nat -j ACCEPT
-A FORWARD -p tcp -j f2b-recidive
-A f2b-sshd -j RETURN
-A f2b-recidive -j RETURN
COMMIT
# Completed on Wed Jul 17 11:10:37 2019
# Generated by xtables-save v1.8.3 on Wed Jul 17 11:10:37 2019
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -j MASQUERADE
COMMIT
# Completed on Wed Jul 17 11:10:37 2019
# Generated by xtables-save v1.8.3 on Wed Jul 17 11:10:37 2019
*mangle
:PREROUTING ACCEPT [248302:1063534761]
:INPUT ACCEPT [247677:1063489218]
:FORWARD ACCEPT [538:39768]
:OUTPUT ACCEPT [228241:18791314]
:POSTROUTING ACCEPT [228779:18831082]
-A POSTROUTING -o lxc-bridge-nat -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
COMMIT
# Completed on Wed Jul 17 11:10:37 2019
# Generated by xtables-save v1.8.3 on Wed Jul 17 11:10:37 2019
*raw
:PREROUTING ACCEPT [3012:193211]
:OUTPUT ACCEPT [2048:239550]
COMMIT
# Completed on Wed Jul 17 11:10:37 2019
# Warning: iptables-legacy tables present, use iptables-legacy-save to see them

I have also tried the tcpdump -i eno1 -n icmp command to see if there was any issue related to the IP packet and here is the output (It has been run on the host side during a ping to 8.8.8.8 on the container side):

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eno1, link-type EN10MB (Ethernet), capture size 262144 bytes
11:12:38.938243 IP 192.168.100.78 > 8.8.8.8: ICMP echo request, id 1137, seq 0, length 64
11:12:39.939343 IP 192.168.100.78 > 8.8.8.8: ICMP echo request, id 1137, seq 1, length 64
11:12:40.940554 IP 192.168.100.78 > 8.8.8.8: ICMP echo request, id 1137, seq 2, length 64
11:12:41.941748 IP 192.168.100.78 > 8.8.8.8: ICMP echo request, id 1137, seq 3, length 64
11:12:42.942939 IP 192.168.100.78 > 8.8.8.8: ICMP echo request, id 1137, seq 4, length 64

Has you can see the packet goes on the Internet through the public interface but it is using the IP address of the container and not the host.

So I assume my issue is related to that but I don't know why it does that and how to fix it.

If you need any more information in order to solve my problem, feel free to ask.

------- EDIT: add iptables raw output iptables -nvL

Chain INPUT (policy ACCEPT 251K packets, 1064M bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  lxc-bridge-nat *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53
    6   430 ACCEPT     udp  --  lxc-bridge-nat *       0.0.0.0/0            0.0.0.0/0            udp dpt:53
    0     0 ACCEPT     tcp  --  lxc-bridge-nat *       0.0.0.0/0            0.0.0.0/0            tcp dpt:67
   16  5295 ACCEPT     udp  --  lxc-bridge-nat *       0.0.0.0/0            0.0.0.0/0            udp dpt:67
 219K 1062M f2b-recidive  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           
 215K 1046M f2b-sshd   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            multiport dports 22

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  *      lxc-bridge-nat  0.0.0.0/0            0.0.0.0/0           
  131 10860 ACCEPT     all  --  lxc-bridge-nat *       0.0.0.0/0            0.0.0.0/0           
    0     0 f2b-recidive  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 233K packets, 19M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain f2b-sshd (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 214K 1046M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain f2b-recidive (2 references)
 pkts bytes target     prot opt in     out     source               destination         
  874 52624 REJECT     all  --  *      *       106.75.71.124        0.0.0.0/0            reject-with icmp-port-unreachable
  246 14760 REJECT     all  --  *      *       51.75.23.87          0.0.0.0/0            reject-with icmp-port-unreachable
 218K 1062M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           
# Warning: iptables-legacy tables present, use iptables-legacy to see them

iptables -nvL -t nat

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MASQUERADE  all  --  *      *       192.168.100.0/24    !192.168.100.0/24    

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
# Warning: iptables-legacy tables present, use iptables-legacy to see them
Forbinn
  • 11
  • 1
  • 5
  • `-A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -j MASQUERADE` is missing `-t nat`: `-t nat -A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -j MASQUERADE`. `net.ipv4.ip_forward` is set to 1? – Lenniey Jul 17 '19 at 11:42
  • `cat /proc/sys/net/ipv4/ip_forward` -> `1`. The `MASQUERADE` rules is present in iptables – Forbinn Jul 17 '19 at 11:49
  • Sorry, I missed the `*nat`. I'd add `-o eno1` when using MASQUERADE. Do you have a static IP on `eno1`? Then use `SNAT` instead of MASQUERADE. Other than that, this all looks OK to me. Is there some iptables rule present that you don't know of? What's the output of `iptables -nvL` and `iptables -nvL -t nat`? – Lenniey Jul 17 '19 at 12:03
  • No problem. The `MASQUERADE` rule has been automatically created by `lxc-net` service so I don't think it is the issue but I will try anyway. I gave you the whole `iptables-save` output without removing anything. I add the output you've asked in the question anyway. – Forbinn Jul 17 '19 at 12:31
  • I just tried with the `-o eno1` option (and of course I have removed the previous rule) and nothing changes, still no ping to `8.8.8.8` – Forbinn Jul 17 '19 at 12:34
  • I tried to use SNAT rule (`iptables -t nat -A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -j SNAT --to-source `) instead of the `MASQUERADE` one and nothing changes, still no Internet access inside the container. – Forbinn Jul 18 '19 at 08:05
  • Like I said, I can't see anything wrong, then, sorry. – Lenniey Jul 18 '19 at 08:21

1 Answers1

0

Ok, I found the issue (thank to tomp on https://discuss.linuxcontainers.org). By default lxc-net uses iptables to create rules which is fine but not working on my system because # Warning: iptables-legacy tables present, use iptables-legacy to see them.

So after executing the command iptables-legacy -t nat -A POSTROUTING -s 192.168.100.0/24 ! -d 192.168.100.0/24 -j MASQUERADE the system now works fine.

I now have two little more questions regarding this issue: why is this the case ? And how can I make this work using the non-legacy system ?

Forbinn
  • 11
  • 1
  • 5