3

Right now I have an Ubuntu 16 EC2 instance set up with ipv6 connectivity. I specifically am just trying to SSH and OpenVPN via IPV6. The following work:

  • ipv4 SSH and OpenVPN connections
  • ipv6 outbound connections, such as ping6 and curl
  • ipv6 inbound connections if I am connected to the server hosted VPN, which is an ipv4 connection

I checked the security groups, double checked and went through every step of the AWS IPV6 migration guide, and cleared all the ip6tables. I have not made any progress towards fixing this issue.

Here is what happens when I try to SSH outside the VPN:

$ ssh ubuntu@example.com -i "example.key" -6 -v
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to example.com [2600:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx] port 22.
debug1: connect to address 2600:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx port 22: Resource temporarily unavailable
ssh: connect to host example.com port 22: Resource temporarily unavailable

When I connect via IPV4 on OpenVPN, then SSH via IPV6:

$ ssh ubuntu@2001:db8:ee00:abcd::1 -i "example.key" -6 -v
OpenSSH_6.6.1, OpenSSL 1.0.1f 6 Jan 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 2001:db8:ee00:abcd::1 [2001:db8:ee00:abcd::1] port 22.
debug1: Connection established.
debug1: identity file example.key type -1
debug1: identity file example.key-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.8
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.2
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.2 pat OpenSSH* compat 0x04000000
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-sha1-etm@openssh.com none
debug1: kex: client->server aes128-ctr hmac-sha1-etm@openssh.com none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA bd:a7:ac:dd:37:98:c0:8f:7a:f6:e7:e8:20:05:36:48
The authenticity of host '2001:db8:ee00:abcd::1 (2001:db8:ee00:abcd::1)' can't be established.
ECDSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '2001:db8:ee00:abcd::1' (ECDSA) to the list of known hosts.
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: example.key
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey).
Authenticated to 2001:db8:ee00:abcd::1 ([2001:db8:ee00:abcd::1]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
setsockopt IPV6_TCLASS 16: Operation not permitted:
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-1035-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

6 packages can be updated.
0 updates are security updates.


Last login: Sat Sep 30 04:32:44 2017 from xxx.xxx.xxx.xxx

This is the security group for this instance:

Security groups for EC2

Edit 1: Here is a tcpdump. It appears the server is seeing the packets. 

17:26:33.761004 IP6 (flowlabel 0x93c4a, hlim 50, next-header TCP (6) payload length: 32) 2601:my:home:ipv6:addr:xxxx:xxxx:xxxx.64941 > 2600:my:server:ipv6:addr:yyyy:yyyy:yyyy.ssh: Flags [S], cksum 0x20be (correct), seq 2537279844, win 64800, options [mss 1440,nop,wscale 8,nop,nop,sackOK], length 0
17:26:36.761425 IP6 (flowlabel 0x93c4a, hlim 50, next-header TCP (6) payload length: 32) 2601:my:home:ipv6:addr:xxxx:xxxx:xxxx.64941 > 2600:my:server:ipv6:addr:yyyy:yyyy:yyyy.ssh: Flags [S], cksum 0x20be (correct), seq 2537279844, win 64800, options [mss 1440,nop,wscale 8,nop,nop,sackOK], length 0
17:26:42.260168 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 56) fe80::b7:e1ff:fee7:e95e > ip6-allnodes: [icmp6 sum ok] ICMP6, router advertisement, length 56
    hop limit 64, Flags [managed], pref medium, router lifetime 1800s, reachable time 0s, retrans time 0s
      source link-address option (1), length 8 (1): 02:b7:e1:e7:e9:5e
        0x0000:  02b7 e1e7 e95e
      prefix info option (3), length 32 (4): 2600:my:server:ipv6::/64, Flags [none], valid time infinity, pref. time infinity
        0x0000:  4000 ffff ffff ffff ffff 0000 0000 2600
        0x0010:  1f1c 0c41 b120 0000 0000 0000 0000
17:26:42.761137 IP6 (flowlabel 0x93c4a, hlim 50, next-header TCP (6) payload length: 32) 2601:my:home:ipv6:addr:xxxx:xxxx:xxxx.64941 > 2600:my:server:ipv6:addr:yyyy:yyyy:yyyy.ssh: Flags [S], cksum 0x20be (correct), seq 2537279844, win 64800, options [mss 1440,nop,wscale 8,nop,nop,sackOK], length 0
17:26:52.260303 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 56) fe80::b7:e1ff:fee7:e95e > ip6-allnodes: [icmp6 sum ok] ICMP6, router advertisement, length 56
    hop limit 64, Flags [managed], pref medium, router lifetime 1800s, reachable time 0s, retrans time 0s
      source link-address option (1), length 8 (1): 02:b7:e1:e7:e9:5e
        0x0000:  02b7 e1e7 e95e
      prefix info option (3), length 32 (4): 2600:my:server:ipv6::/64, Flags [none], valid time infinity, pref. time infinity
        0x0000:  4000 ffff ffff ffff ffff 0000 0000 2600
        0x0010:  1f1c 0c41 b120 0000 0000 0000 0000

Edit 2: Here is a tcpdump after disabling miredo. However, ping6 now returns an error connect: Network is unreachable 

17:52:44.291012 IP6 (flowlabel 0xc8b54, hlim 50, next-header TCP (6) payload length: 32) 2601:my:home:ipv6:addr:xxxx:xxxx:xxxx.65166 > 2600:my:server:ipv6:addr:yyyy:yyyy:yyyy.ssh: Flags [S], cksum 0xf081 (correct), seq 4210466052, win 64800, options [mss 1440,nop,wscale 8,nop,nop,sackOK], length 0
17:52:47.291056 IP6 (flowlabel 0xc8b54, hlim 50, next-header TCP (6) payload length: 32) 2601:my:home:ipv6:addr:xxxx:xxxx:xxxx.65166 > 2600:my:server:ipv6:addr:yyyy:yyyy:yyyy.ssh: Flags [S], cksum 0xf081 (correct), seq 4210466052, win 64800, options [mss 1440,nop,wscale 8,nop,nop,sackOK], length 0
17:52:52.272999 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 56) fe80::b7:e1ff:fee7:e95e > ip6-allnodes: [icmp6 sum ok] ICMP6, router advertisement, length 56
    hop limit 64, Flags [managed], pref medium, router lifetime 1800s, reachable time 0s, retrans time 0s
      source link-address option (1), length 8 (1): 02:b7:e1:e7:e9:5e
        0x0000:  02b7 e1e7 e95e
      prefix info option (3), length 32 (4): 2600:my:server:ipv6::/64, Flags [none], valid time infinity, pref. time infinity
        0x0000:  4000 ffff ffff ffff ffff 0000 0000 2600
        0x0010:  1f1c 0c41 b120 0000 0000 0000 0000
17:52:53.298882 IP6 (flowlabel 0xc8b54, hlim 50, next-header TCP (6) payload length: 32) 2601:my:home:ipv6:addr:xxxx:xxxx:xxxx.65166 > 2600:my:server:ipv6:addr:yyyy:yyyy:yyyy.ssh: Flags [S], cksum 0xf081 (correct), seq 4210466052, win 64800, options [mss 1440,nop,wscale 8,nop,nop,sackOK], length 0
17:53:02.273102 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 56) fe80::b7:e1ff:fee7:e95e > ip6-allnodes: [icmp6 sum ok] ICMP6, router advertisement, length 56
    hop limit 64, Flags [managed], pref medium, router lifetime 1800s, reachable time 0s, retrans time 0s
      source link-address option (1), length 8 (1): 02:b7:e1:e7:e9:5e
        0x0000:  02b7 e1e7 e95e
      prefix info option (3), length 32 (4): 2600:my:server:ipv6::/64, Flags [none], valid time infinity, pref. time infinity
        0x0000:  4000 ffff ffff ffff ffff 0000 0000 2600
        0x0010:  1f1c 0c41 b120 0000 0000 0000 0000
17:53:12.273190 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 56) fe80::b7:e1ff:fee7:e95e > ip6-allnodes: [icmp6 sum ok] ICMP6, router advertisement, length 56
    hop limit 64, Flags [managed], pref medium, router lifetime 1800s, reachable time 0s, retrans time 0s
      source link-address option (1), length 8 (1): 02:b7:e1:e7:e9:5e
        0x0000:  02b7 e1e7 e95e
      prefix info option (3), length 32 (4): 2600:my:server:ipv6::/64, Flags [none], valid time infinity, pref. time infinity
        0x0000:  4000 ffff ffff ffff ffff 0000 0000 2600
        0x0010:  1f1c 0c41 b120 0000 0000 0000 0000
17:53:22.273260 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 56) fe80::b7:e1ff:fee7:e95e > ip6-allnodes: [icmp6 sum ok] ICMP6, router advertisement, length 56
    hop limit 64, Flags [managed], pref medium, router lifetime 1800s, reachable time 0s, retrans time 0s
      source link-address option (1), length 8 (1): 02:b7:e1:e7:e9:5e
        0x0000:  02b7 e1e7 e95e
      prefix info option (3), length 32 (4): 2600:my:server:ipv6::/64, Flags [none], valid time infinity, pref. time infinity
        0x0000:  4000 ffff ffff ffff ffff 0000 0000 2600
        0x0010:  1f1c 0c41 b120 0000 0000 0000 0000
Jack Cole
  • 171
  • 7
  • Have you tried disabling/removing iptables all together? Looks like a firewall issue still. Maybe a tcpdump on the offending system would help. – Appleoddity Sep 30 '17 at 15:22
  • I added a tcpdump – Jack Cole Sep 30 '17 at 17:34
  • Also I flushed all the ip6tables. There is no ip6tables or iptables service installed. – Jack Cole Sep 30 '17 at 17:41
  • Thanks. The server is not completing the TCP handshake. So, it sees the incoming SYN packet but doesn't respond with the SYN ACK. So AWS spears configured fine. For some reason your server is not listening on that address or port or a firewall is blocking it. – Appleoddity Sep 30 '17 at 17:42
  • I've disabled ufw. What else could the server be doing to firewall it? – Jack Cole Sep 30 '17 at 17:46
  • OK So I just remembered I had miredo installed, because I couldn't get ipv6 working without it. I updated my post to reflect what happened after I disabled miredo. Not sure if that has anything to do with it. – Jack Cole Sep 30 '17 at 17:56

1 Answers1

2

The issue was due to incorrect permissions for /etc/network/interfaces.d/60-default-with-ipv6.cfg. It was set to only the user could read the file. I fixed the issue by running the following commands:

$ sudo chmod go+r /etc/network/interfaces.d/60-default-with-ipv6.cfg
$ sudo ifdown eth0 ; sudo ifup eth0

IPV6 now fully works inbound and outbound. OpenVPN does not work with ipv6 still, but that is another issue.

Jack Cole
  • 171
  • 7