1

In our setup we have to access EC2 instances via a bastionhost. This bastionhost is connected to an Active Directory where it validates the credentials of the user who connects. From the bastion host the connection is being made with an ssh key which I mustn't store on the bastionhost but rather "forward". So the connection goes this way:

A (my PC, has key) ---- aduser/adpass ---> B (bastion host) ---- key ---> C (EC2 target)

I did set up new Ubuntu eoan (19.10) server with Ansible 2.8.1 (via apt install ansible and from the official Ubuntu repo).

In my ~/ansible I created two files. One is the hosts file:

[gatewayed]
foo1 ansible_host=ip-1-2-3-4.eu-central-1.compute.internal

(IP address redacted)

The other group_vars/gatewayed.yml:

ansible_ssh_common_args: '-o ProxyCommand="ssh -W %h:%p -q aduser@bastion.my.company"'

Into ~/.ssh/id_rsa I copied the private key (and chmoded it to rw for my only) I have to use for connecting to the target server on AWS. It is the only key stored there.

When I run ansible -vvvv all -i ~/ansible/hosts -a "uname -a" I am queried for the password which I enter:

aduser@bastion.my.company's password:

But the connection fails with this output (taken from the msg field of the response, formatted for better legibility):

"msg": "Failed to connect to the host via ssh: OpenSSH_8.0p1 Ubuntu-6build1, OpenSSL 1.1.1c  28 May 2019
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket \"/home/ubuntu/.ansible/cp/a1ffe79391\" does not exist
debug1: Executing proxy command: exec ssh -W ip-1-2-3-4.eu-central-1.compute.internal:22 -q aduser@bastion.my.company
debug3: timeout: 10000 ms remain after connect
debug1: identity file /home/ubuntu/.ssh/id_rsa type -1
debug1: identity file /home/ubuntu/.ssh/id_rsa-cert type -1
debug1: identity file /home/ubuntu/.ssh/id_dsa type -1
debug1: identity file /home/ubuntu/.ssh/id_dsa-cert type -1
debug1: identity file /home/ubuntu/.ssh/id_ecdsa type -1
debug1: identity file /home/ubuntu/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/ubuntu/.ssh/id_ed25519 type -1
debug1: identity file /home/ubuntu/.ssh/id_ed25519-cert type -1
debug1: identity file /home/ubuntu/.ssh/id_xmss type -1
debug1: identity file /home/ubuntu/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.0p1 Ubuntu-6build1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug2: fd 5 setting O_NONBLOCK
debug2: fd 4 setting O_NONBLOCK
debug1: Authenticating to ip-1-2-3-4.eu-central-1.compute.internal:22 as 'ubuntu'
debug3: hostkeys_foreach: reading file \"/home/ubuntu/.ssh/known_hosts\"
debug3: record_hostkey: found key type ECDSA in file /home/ubuntu/.ssh/known_hosts:3
debug3: load_hostkeys: loaded 1 keys from ip-1-2-3-4.eu-central-1.compute.internal
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: zlib@openssh.com,zlib,none
debug2: compression stoc: zlib@openssh.com,zlib,none
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:szq8W0Cu8dTQwEtiHXPm62EmQZJtUsXkyxulS9UR0zw
debug3: hostkeys_foreach: reading file \"/home/ubuntu/.ssh/known_hosts\"
debug3: record_hostkey: found key type ECDSA in file /home/ubuntu/.ssh/known_hosts:3
debug3: load_hostkeys: loaded 1 keys from ip-1-2-3-4.eu-central-1.compute.internal
debug1: Host 'ip-1-2-3-4.eu-central-1.compute.internal' is known and matches the ECDSA host key.
debug1: Found key in /home/ubuntu/.ssh/known_hosts:3
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /home/ubuntu/.ssh/id_rsa 
debug1: Will attempt key: /home/ubuntu/.ssh/id_dsa 
debug1: Will attempt key: /home/ubuntu/.ssh/id_ecdsa 
debug1: Will attempt key: /home/ubuntu/.ssh/id_ed25519 
debug1: Will attempt key: /home/ubuntu/.ssh/id_xmss 
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic
debug3: preferred gssapi-with-mic,gssapi-keyex,hostbased,publickey
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: gssapi-keyex,hostbased,publickey
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information\nNo Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000)\n\n
debug1: Unspecified GSS failure.  Minor code may provide more information\nNo Kerberos credentials available (default cache: FILE:/tmp/krb5cc_1000)\n\n
debug2: we did not send a packet, disable method
debug3: authmethod_lookup gssapi-keyex
debug3: remaining preferred: hostbased,publickey
debug3: authmethod_lookup publickey
debug3: remaining preferred: ,publickey
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /home/ubuntu/.ssh/id_rsa
debug3: sign_and_send_pubkey: RSA SHA256:nX/Xap0M0+mhKEb+AEYLpsF2qVftrfnelOyNgQGShcw
debug3: sign_and_send_pubkey: signing using rsa-sha2-512
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Trying private key: /home/ubuntu/.ssh/id_dsa
debug3: no such identity: /home/ubuntu/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /home/ubuntu/.ssh/id_ecdsa
debug3: no such identity: /home/ubuntu/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /home/ubuntu/.ssh/id_ed25519
debug3: no such identity: /home/ubuntu/.ssh/id_ed25519: No such file or directory
debug1: Trying private key: /home/ubuntu/.ssh/id_xmss
debug3: no such identity: /home/ubuntu/.ssh/id_xmss: No such file or directory
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
ubuntu@ip-1-2-3-4.eu-central-1.compute.internal: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).",
    "unreachable": true

I tried options -J and -A with ssh, like suggested here, but this didn't solve the problem. The same connection works flawlessly from the same computer when I use Mobaxterm, but there configuration is much easier (at least to me) because I only have to fill in the fields in the GUI ;-)

Marged
  • 213
  • 1
  • 2
  • 10

1 Answers1

0

I found and combined several sources by Googling (sorry, I don't remember the links. If I would remember them I would mention them here to give credit).

Basically I removed the gatewayed group and the group var, Ansible doesn't need to know if the server can be reached directly or not.

Then I created this ~/.ssh/config file which works fine for ssh, scp and Ansible. In all three tools I just have to specify the easier to remember Host alias:

Host foo1
    User ec2-user
    Hostname ip-1-2-3-4.eu-central-1.compute.internal
    PreferredAuthentications publickey
    IdentityFile ~/.ssh/id_rsa
    ForwardAgent yes
    ProxyCommand ssh jump -W %h:%p

Host jump
    User aduser
    HostName bastion.my.company
    IdentityFile ~/.ssh/id_rsa

For Ansible this works perfectly, I can either use playbooks or run ad hoc commands like this:

ansible aws --ask-pass -i ~/ansible/hosts -a "uname"

In my scenario it is important to add --ask-pass because only one of the connections is secured by private key, the other needs the password of the aduser.

Marged
  • 213
  • 1
  • 2
  • 10