1

I'm using Ubuntu 18.04 LTS on AWS and I'm trying to setup split tunneling using commercial VPN (PIA).

I'm unable to use their .opvn configuration out of the box because when I do so my SSH connection drops and I have to restart the server in order to get back, so I added pull-filter ignore redirect-gateway (which is a modern version of route-nopull).

My .opvn file (NYC.opvn) contains this:

client
dev tun
proto udp
remote us-newyorkcity.privateinternetaccess.com 1198
resolv-retry infinite
nobind
persist-key
persist-tun
cipher aes-128-cbc
auth sha1
tls-client
remote-cert-tls server
pull-filter ignore redirect-gateway

auth-user-pass login.conf
compress
verb 1
reneg-sec 0

I start the tunnel with sudo openvpn NYC.opvn and get this:

Wed Nov 20 17:34:52 2019 OpenVPN 2.4.4 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on May 14 2019
Wed Nov 20 17:34:52 2019 library versions: OpenSSL 1.1.1d  10 Sep 2019, LZO 2.08
Wed Nov 20 17:34:52 2019 TCP/UDP: Preserving recently used remote address: [AF_INET]209.95.50.11:1198
Wed Nov 20 17:34:52 2019 UDP link local: (not bound)
Wed Nov 20 17:34:52 2019 UDP link remote: [AF_INET]209.95.50.11:1198
Wed Nov 20 17:34:52 2019 [8d26667dabcc2a9cc7b10009813a306a] Peer Connection Initiated with [AF_INET]209.95.50.11:1198
Wed Nov 20 17:34:53 2019 TUN/TAP device tun0 opened
Wed Nov 20 17:34:53 2019 do_ifconfig, tt->did_ifconfig_ipv6_setup=0
Wed Nov 20 17:34:53 2019 /sbin/ip link set dev tun0 up mtu 1500
Wed Nov 20 17:34:53 2019 /sbin/ip addr add dev tun0 local 10.43.11.6 peer 10.43.11.5
Wed Nov 20 17:34:53 2019 Initialization Sequence Completed

However, I don't have internet access on tun0:

curl --interface tun0 -v ipinfo.io
* Rebuilt URL to: ipinfo.io/
*   Trying 216.239.38.21...
* TCP_NODELAY set
* Local Interface tun0 is ip 10.43.11.6 using address family 2
* Local port: 0

and it just hangs (curl --interface ens5 -v ipinfo.io works fine).

About the network (after VPN is started):

route -n

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.31.32.1     0.0.0.0         UG    100    0        0 ens5
10.43.11.1      10.43.11.5      255.255.255.255 UGH   0      0        0 tun0
10.43.11.5      0.0.0.0         255.255.255.255 UH    0      0        0 tun0
172.31.32.0     0.0.0.0         255.255.240.0   U     0      0        0 ens5
172.31.32.1     0.0.0.0         255.255.255.255 UH    100    0        0 ens5


ip addr show

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 0e:a7:57:4c:36:ab brd ff:ff:ff:ff:ff:ff
    inet 172.31.47.214/20 brd 172.31.47.255 scope global dynamic ens5
       valid_lft 3359sec preferred_lft 3359sec
    inet6 fe80::ca7:57ff:fe4c:36ab/64 scope link 
       valid_lft forever preferred_lft forever
19: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 100
    link/none 
    inet 10.43.11.6 peer 10.43.11.5/32 scope global tun0
       valid_lft forever preferred_lft forever
    inet6 fe80::2444:7d65:81eb:af01/64 scope link stable-privacy 
       valid_lft forever preferred_lft forever

How do I get internet working on tun0 so I can use it with cURL?

Thanks alot

EDIT

A network admin told me I have to do the following:

  • create one more routing table
  • add default route into that custom table 0.0.0.0/0 via 10.43.11.5
  • add rule for packets originating from tun0 to lookup routes from that new routing table

So I did:

(note: tun0 IPs have changed since the original posting and are now inet 10.55.10.6 netmask 255.255.255.255 destination 10.55.10.5)

echo "1000 vpn" >> /etc/iproute2/rt_tables
ip route add default via 10.55.10.5 dev tun0 table vpn
ip rule add iif tun0 lookup vpn
ip route add 255.255.255.255 dev tun0 proto kernel src 10.55.10.6 table vpn

Unfortunately, cURL still hangs:

curl -v --interface tun0  http://ipinfo.io
* Rebuilt URL to: http://ipinfo.io/
*   Trying 216.239.36.21...
* TCP_NODELAY set
* Local Interface tun0 is ip 10.55.10.6 using address family 2
* Local port: 0

Here's more debugging data:

ubuntu@ip-172-31-47-214:~$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.31.32.1     0.0.0.0         UG    100    0        0 ens5
10.55.10.1      10.55.10.5      255.255.255.255 UGH   0      0        0 tun0
10.55.10.5      0.0.0.0         255.255.255.255 UH    0      0        0 tun0
172.31.32.0     0.0.0.0         255.255.240.0   U     0      0        0 ens5
172.31.32.1     0.0.0.0         255.255.255.255 UH    100    0        0 ens5
ubuntu@ip-172-31-47-214:~$ ifconfig
ens5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9001
        inet 172.31.47.214  netmask 255.255.240.0  broadcast 172.31.47.255
        inet6 fe80::ca7:57ff:fe4c:36ab  prefixlen 64  scopeid 0x20<link>
        ether 0e:a7:57:4c:36:ab  txqueuelen 1000  (Ethernet)
        RX packets 1188  bytes 117537 (117.5 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 841  bytes 109924 (109.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 168  bytes 13034 (13.0 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 168  bytes 13034 (13.0 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

tun0: flags=4305<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST>  mtu 1500
        inet 10.55.10.6  netmask 255.255.255.255  destination 10.55.10.5
        inet6 fe80::28a:e5f2:9cdf:b63  prefixlen 64  scopeid 0x20<link>
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 100  (UNSPEC)
        RX packets 10  bytes 600 (600.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 13  bytes 684 (684.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ubuntu@ip-172-31-47-214:~$ ip rule show
0:  from all lookup local 
32765:  from all iif tun0 lookup vpn 
32766:  from all lookup main 
32767:  from all lookup default
poige
  • 9,448
  • 2
  • 25
  • 52
The Onin
  • 41
  • 10
  • You need a default route that points to the IP address of the other end of the tunnel's tunnel address. – David Schwartz Nov 20 '19 at 21:12
  • @DavidSchwartz I have no idea (i) what's my other end's tunnel address (ii) what's a default route (it *sounds* like if I use a default route, I'll be undoing `pull-filter ignore redirect-gateway`). – The Onin Nov 20 '19 at 21:50
  • @DavidSchwartz I think I figured out what the IP address on the other end of tunnel is (`10.79.10.5` for the latest session, but it seems to change every time), so I ran `sudo ip route add 10.79.10.0/24 via 10.79.10.5 dev tun0` but `curl -v --interface tun0 http://ipinfo.io` still hangs. – The Onin Nov 22 '19 at 18:37

2 Answers2

2

man ip-rule: "…

oif NAME — select the outgoing device to match. The outgoing interface is only available for packets originating from local sockets that are bound to a device.

…"

Hence, instead of ip rule add iif tun0 lookup vpn you should use:

ip rule add oif tun0 lookup vpn

But in fact it won't work because output interface will be selected by primary route table before. So, the only feasible option is to use ip rule based on source IP. For e. g., if you're sure VPN's IP will stay in network 10/8 as 10.43.11.6 does it would be as simple as

ip rule add from 10.0.0.0/8 lookup vpn
poige
  • 9,448
  • 2
  • 25
  • 52
  • (I have an idea how to make this more precise, but let me know first if this one worked for you). – poige Nov 23 '19 at 21:08
  • This works for me - thanks a lot! Do you have any ideas on what would be the best way to automate connecting to VPN and adding those 2 routes on system startup? 1. start the VPN: `sudo openvpn /etc/openvpn/NYC.opvn` 2. fetch the peer IP - by watching the connection logs or ifconfig 3. `ip route add default via 10.45.10.5 dev tun0 table vpn` 4. `ip rule add from 10.0.0.0/8 lookup vpn`. enjoy the spoils :) – The Onin Nov 25 '19 at 17:20
  • I will. :) As per additional questions: you don't need to find peer's IP, since it's point-to-point only dev name is sufficient. I'd probably install fresh enough Bird or Quagga that should be able to re-instate such a route into a custom routing table as soon as interface comes up (although they would stick to `tun0` name which might be or might be not suitable in some cases). – poige Nov 25 '19 at 17:40
  • If you're comfortable with shell scripting, there's a tinkering. Unexpectedly, quick WEB search didn't reveal anything like ready "`inotify`-handlers for network", so poor man's solution could be *running a script that would parse `ip monitor route` output, and run another script to install route into a table*. You can try running that command and flip connection up-n-down, you shall get the idea. There's https://linux.die.net/man/8/netplugd but I never used it, and ain't have no clue if it'd fit. – poige Nov 25 '19 at 17:44
  • also see this: https://stackoverflow.com/questions/2261759/get-notified-about-network-interface-change-on-linux – poige Nov 25 '19 at 17:52
0

Your IP address on that computer is a "private IP address". the default route "Destination: 0.0.0.0" is also using a "private IP address". and your VPN is using a different "private IP address".

the difference is, probably, that your default route, which is on ens5, uses a technology similar to NAT that translates that "private IP address" to a "public IP address".

when you use the tun0 device you are going to a server that does not implement NAT.

so, if you have root access to the machine that is at "10.43.11.1" you can set up NAT there, and it, too, will translate the "private IP address" to a "public IP address".

Hope this helps.

if not please feel free to question.

armadefuego
  • 11
  • 1
  • Hey, I actually do think that tun0 implements NAT on its counterpart - as I said in the original post, I'm using a commercial VPN (PIA VPN) in order to encrypt traffic from the server to the internet and when I check whatsmyip using tun0 (on my local computer where I use an app, making sure I connect to the same region) the private and public IPs are completely different. – The Onin Nov 25 '19 at 16:44