I'm playing around with StrongSwan as a VPN provider for a few personal projects, so I'm running it on the same server as the other things I'm doing. It's all set-up and working, however I'm having an issue accessing services (e.g. nginx) on the same server.
The following two options are present in the ipsec.conf file:
leftfirewall=yes
lefthostaccess=yes
When I can connect to the VPN I can see the necessary routing and firewall rules added as follows:
Firewall
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
169 21051 ACCEPT all -- eth0 any 10.11.12.1 anywhere policy match dir in pol ipsec reqid 2 proto esp
133 34163 ACCEPT all -- any eth0 anywhere 10.11.12.1 policy match dir out pol ipsec reqid 2 proto esp
Route list (table 220)
10.11.12.1 via [server ip] dev eth0 proto static
When I check the packets that are coming in and being dropped, I can see that the source IP isn't that of the VPN (i.e. the traffic isn't being routed via the VPN). I know I'm connected (from ipsec statusall and also by checking "what's my ip").
So the issue seems to be that the traffic isn't being routed to the VPN. Is this a server-side issue, or an issue with the device connected to the VPN? All web traffic is set to go through the VPN (which is verified by checking what IP is seen by other websites).
I have seen this similar question, but the answer doesn't actually provide the correct fix as far as I can see.
As per ecdsa's suggestion, here's a bit more info and clarification.
The remote user (with public IP address A.A.A.A) connects to the VPN server (with public IP address B.B.B.B). On the same server, there is an nginx server running. The remote user can access that server from a TLD which points to B.B.B.B.
When connected to the VPN (as shown below), the user is correctly assigned an IP address from the virtual IP pool. If the user runs the following command: curl icanhazip.com the output is the IP address of the VPN server (B.B.B.B).
Now, when connected to the VPN, if the user tries to access the nginx server via the TLD, the packets don't seem to be routed via the VPN. So my problem is trying to get the packets destined for the same server that the VPN sits on, to be routed via the VPN.
ipsec statusall
Status of IKE charon daemon (strongSwan 5.3.5, Linux 4.4.0-119-generic, x86_64):
uptime: 111 seconds, since Apr 09 12:15:11 2018
malloc: sbrk 1642496, mmap 0, used 581936, free 1060560
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 2
loaded plugins: charon test-vectors aes rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp agent xcbc hmac gcm attr kernel-netlink resolve socket-default connmark farp stroke updown eap-identity eap-sim eap-sim-pcsc eap-aka eap-aka-3gpp2 eap-simaka-pseudonym eap-simaka-reauth eap-md5 eap-gtc eap-mschapv2 eap-dynamic eap-radius eap-tls eap-ttls eap-peap eap-tnc xauth-generic xauth-eap xauth-pam xauth-noauth tnc-tnccs tnccs-20 tnccs-11 tnccs-dynamic dhcp lookip error-notify certexpire led addrblock unity
Virtual IP pools (size/online/offline):
10.11.12.0/24: 254/1/0
Listening IP addresses:
SERVER_IP
Connections:
ikev2-vpn: %any...%any IKEv2, dpddelay=300s
ikev2-vpn: local: [vpn.TLD.co.uk] uses public key authentication
ikev2-vpn: cert: "CN=vpn.TLD.co.uk"
ikev2-vpn: remote: uses EAP_MSCHAPV2 authentication with EAP identity '%any'
ikev2-vpn: child: 0.0.0.0/0 === dynamic TUNNEL, dpdaction=clear
Security Associations (1 up, 0 connecting):
ikev2-vpn[1]: ESTABLISHED 4 seconds ago, SERVER_IP[vpn.TLD.co.uk]...REMOTE_IP[vpn.TLD.co.uk]
ikev2-vpn[1]: Remote EAP identity: USER
ikev2-vpn[1]: IKEv2 SPIs: 38003da03964bd2d_i 83715a1671973e21_r*, rekeying disabled
ikev2-vpn[1]: IKE proposal: AES_GCM_16_256/PRF_HMAC_SHA2_256/ECP_521
ikev2-vpn{1}: INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: c6f1750a_i 00b2ead4_o
ikev2-vpn{1}: AES_GCM_16_256, 174 bytes_i (3 pkts, 2s ago), 502 bytes_o (3 pkts, 2s ago), rekeying disabled
ikev2-vpn{1}: 0.0.0.0/0 === 10.11.12.1/32
Connection Log:
Apr 9 12:59:00 mmstr charon: 11[NET] received packet: from REMOTE_IP[627] to SERVER_IP[500] (300 bytes)
Apr 9 12:59:00 mmstr charon: 11[ENC] parsed IKE_SA_INIT request 0 [ SA KE No N(REDIR_SUP) N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) ]
Apr 9 12:59:00 mmstr charon: 11[IKE] REMOTE_IP is initiating an IKE_SA
Apr 9 12:59:00 mmstr charon: 11[IKE] remote host is behind NAT
Apr 9 12:59:00 mmstr charon: 11[ENC] generating IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(MULT_AUTH) ]
Apr 9 12:59:00 mmstr charon: 11[NET] sending packet: from SERVER_IP[500] to REMOTE_IP[627] (316 bytes)
Apr 9 12:59:00 mmstr charon: 13[NET] received packet: from REMOTE_IP[19603] to SERVER_IP[4500] (352 bytes)
Apr 9 12:59:00 mmstr charon: 13[ENC] unknown attribute type (25)
Apr 9 12:59:00 mmstr charon: 13[ENC] parsed IKE_AUTH request 1 [ IDi N(INIT_CONTACT) N(MOBIKE_SUP) IDr CPRQ(ADDR DHCP DNS MASK ADDR6 DHCP6 DNS6 (25)) N(ESP_TFC_PAD_N) N(NON_FIRST_FRAG) SA TSi TSr N(EAP_ONLY) ]
Apr 9 12:59:00 mmstr charon: 13[IKE] initiating EAP_IDENTITY method (id 0x00)
Apr 9 12:59:00 mmstr charon: 13[IKE] received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding
Apr 9 12:59:00 mmstr charon: 13[IKE] peer supports MOBIKE
Apr 9 12:59:00 mmstr charon: 13[IKE] authentication of 'vpn.TLD.co.uk' (myself) with RSA signature successful
Apr 9 12:59:00 mmstr charon: 13[IKE] sending end entity cert "CN=vpn.TLD.co.uk"
Apr 9 12:59:00 mmstr charon: 13[ENC] generating IKE_AUTH response 1 [ IDr CERT AUTH EAP/REQ/ID ]
Apr 9 12:59:00 mmstr charon: 13[ENC] splitting IKE message with length of 2415 bytes into 5 fragments
Apr 9 12:59:00 mmstr charon: 13[ENC] generating IKE_AUTH response 1 [ EF(1/5) ]
Apr 9 12:59:00 mmstr charon: 13[ENC] generating IKE_AUTH response 1 [ EF(2/5) ]
Apr 9 12:59:00 mmstr charon: 13[ENC] generating IKE_AUTH response 1 [ EF(3/5) ]
Apr 9 12:59:00 mmstr charon: 13[ENC] generating IKE_AUTH response 1 [ EF(4/5) ]
Apr 9 12:59:00 mmstr charon: 13[ENC] generating IKE_AUTH response 1 [ EF(5/5) ]
Apr 9 12:59:00 mmstr charon: 13[NET] sending packet: from SERVER_IP[4500] to REMOTE_IP[19603] (544 bytes)
Apr 9 12:59:00 mmstr charon: message repeated 3 times: [ 13[NET] sending packet: from SERVER_IP[4500] to REMOTE_IP[19603] (544 bytes)]
Apr 9 12:59:00 mmstr charon: 13[NET] sending packet: from SERVER_IP[4500] to REMOTE_IP[19603] (487 bytes)
Apr 9 12:59:00 mmstr charon: 12[NET] received packet: from REMOTE_IP[19603] to SERVER_IP[4500] (80 bytes)
Apr 9 12:59:00 mmstr charon: 12[ENC] parsed IKE_AUTH request 2 [ EAP/RES/ID ]
Apr 9 12:59:00 mmstr charon: 12[IKE] received EAP identity 'USER'
Apr 9 12:59:00 mmstr charon: 12[IKE] initiating EAP_MSCHAPV2 method (id 0x10)
Apr 9 12:59:00 mmstr charon: 12[ENC] generating IKE_AUTH response 2 [ EAP/REQ/MSCHAPV2 ]
Apr 9 12:59:00 mmstr charon: 12[NET] sending packet: from SERVER_IP[4500] to REMOTE_IP[19603] (97 bytes)
Apr 9 12:59:00 mmstr charon: 14[NET] received packet: from REMOTE_IP[19603] to SERVER_IP[4500] (128 bytes)
Apr 9 12:59:00 mmstr charon: 14[ENC] parsed IKE_AUTH request 3 [ EAP/RES/MSCHAPV2 ]
Apr 9 12:59:00 mmstr charon: 14[ENC] generating IKE_AUTH response 3 [ EAP/REQ/MSCHAPV2 ]
Apr 9 12:59:00 mmstr charon: 14[NET] sending packet: from SERVER_IP[4500] to REMOTE_IP[19603] (134 bytes)
Apr 9 12:59:00 mmstr charon: 15[NET] received packet: from REMOTE_IP[19603] to SERVER_IP[4500] (72 bytes)
Apr 9 12:59:00 mmstr charon: 15[ENC] parsed IKE_AUTH request 4 [ EAP/RES/MSCHAPV2 ]
Apr 9 12:59:00 mmstr charon: 15[IKE] EAP method EAP_MSCHAPV2 succeeded, MSK established
Apr 9 12:59:00 mmstr charon: 15[ENC] generating IKE_AUTH response 4 [ EAP/SUCC ]
Apr 9 12:59:00 mmstr charon: 15[NET] sending packet: from SERVER_IP[4500] to REMOTE_IP[19603] (65 bytes)
Apr 9 12:59:00 mmstr charon: 16[NET] received packet: from REMOTE_IP[19603] to SERVER_IP[4500] (104 bytes)
Apr 9 12:59:00 mmstr charon: 16[ENC] parsed IKE_AUTH request 5 [ AUTH ]
Apr 9 12:59:00 mmstr charon: 16[IKE] authentication of 'vpn.TLD.co.uk' with EAP successful
Apr 9 12:59:00 mmstr charon: 16[IKE] authentication of 'vpn.TLD.co.uk' (myself) with EAP
Apr 9 12:59:00 mmstr charon: 16[IKE] IKE_SA ikev2-vpn[2] established between SERVER_IP[vpn.TLD.co.uk]...REMOTE_IP[vpn.TLD.co.uk]
Apr 9 12:59:00 mmstr charon: 16[IKE] peer requested virtual IP %any
Apr 9 12:59:00 mmstr charon: 16[IKE] assigning virtual IP 10.11.12.1 to peer 'USER'
Apr 9 12:59:00 mmstr charon: 16[IKE] peer requested virtual IP %any6
Apr 9 12:59:00 mmstr charon: 16[IKE] no virtual IP found for %any6 requested by 'USER'
Apr 9 12:59:00 mmstr charon: 16[IKE] CHILD_SA ikev2-vpn{2} established with SPIs cebf4aaa_i 00674f97_o and TS 0.0.0.0/0 === 10.11.12.1/32
Apr 9 12:59:00 mmstr kernel: [ 2641.863504] audit: type=1400 audit(1523278740.562:21): apparmor="DENIED" operation="open" profile="/usr/lib/ipsec/charon" name="/proc/3338/fd/" pid=3338 comm="charon" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Apr 9 12:59:00 mmstr vpn: + vpn.TLD.co.uk 10.11.12.1/32 == REMOTE_IP -- SERVER_IP == 0.0.0.0/0
Apr 9 12:59:00 mmstr charon: 16[ENC] generating IKE_AUTH response 5 [ AUTH CPRP(ADDR DNS DNS) SA TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) ]
Apr 9 12:59:00 mmstr charon: 16[NET] sending packet: from SERVER_IP[4500] to REMOTE_IP[19603] (233 bytes)
This is the dropped packet
Apr 9 13:04:12 mmstr kernel: [ 2953.930182] IPTables-Dropped: IN=eth0 OUT= MAC=*** SRC=REMOTE_IP DST=SERVER_IP LEN=60 TOS=0x00 PREC=0x00 TTL=53 ID=41666 DF PROTO=TCP SPT=5578 DPT=443 WINDOW=29200 RES=0x00 SYN URGP=0
The important thing to note is that the packet source is the original IP of the remote user (i.e. A.A.A.A).
This is the firewall config iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 10.11.12.1 anywhere policy match dir in pol ipsec reqid 2 proto esp
DROP all -- anywhere anywhere state NEW recent: UPDATE seconds: 60 hit_count: 12 name: DEFAULT side: source mask: 255.255.255.255
all -- anywhere anywhere state NEW recent: SET name: DEFAULT side: source mask: 255.255.255.255
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
DROP all -- anywhere anywhere state INVALID
ACCEPT tcp -- anywhere anywhere tcp dpt:ssh
ACCEPT udp -- anywhere anywhere udp dpt:isakmp
ACCEPT udp -- anywhere anywhere udp dpt:ipsec-nat-t
LOGGING tcp -- anywhere anywhere tcp dpt:http
LOGGING tcp -- anywhere anywhere tcp dpt:https
DROP all -- anywhere anywhere
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- 10.11.12.1 anywhere policy match dir in pol ipsec reqid 2 proto esp
ACCEPT all -- anywhere 10.11.12.1 policy match dir out pol ipsec reqid 2 proto esp
ACCEPT all -- 10.11.12.0/24 anywhere policy match dir in pol ipsec proto esp
ACCEPT all -- anywhere 10.11.12.0/24 policy match dir out pol ipsec proto esp
DROP all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere 10.11.12.1 policy match dir out pol ipsec reqid 2 proto esp
Chain LOGGING (2 references)
target prot opt source destination
LOG all -- anywhere anywhere limit: avg 2/min burst 5 LOG level warning prefix "IPTables-Dropped: "
DROP all -- anywhere anywhere
