0

I am following the Google Cloud Shell REST API documentation here which shows using the users.environments.publicKeys.create method to add a SSH public key to my Cloud Shell.

I have used ssh-keygen to generate a public/private key pair and have successfully added the public key to my Cloud Shell using this API. After doing so I can confirm that the public key is accessible by using the users.environments.get method to get the details for my shell which shows this public key.

Whenever I try to SSH to my Cloud Shell using the corresponding private key I get the error Permission denied (publickey).

  • I have tried using the same public/private key pair to connect to other servers and it works fine, so this would rule out the keys being an issue.
  • I have tried connecting from multiple Windows and Linux clients to rule out any issues with local file permissions, but no luck.
  • I have tried multiple keys but get the same result.

Here is the output of me trying to connect via SSH from a Ubuntu machine:

ssh -i .ssh/id_rsa -p 6000 user@devshell-vm-xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.cloudshell.dev -vvv

OpenSSH_7.2p2 Ubuntu-4ubuntu2.8, OpenSSL 1.0.2g  1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "devshell-vm-xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.cloudshell.dev" port 6000
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to devshell-vm-xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.cloudshell.dev [xx.xx.xx.xx] port 6000.
debug1: Connection established.
debug1: identity file .ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file .ssh/id_rsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.8
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Debian-10+deb9u6
debug1: match: OpenSSH_7.4p1 Debian-10+deb9u6 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to devshell-vm-xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.cloudshell.dev:6000 as 'user'
debug3: put_host_port: [devshell-vm-xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.cloudshell.dev]:6000
debug3: hostkeys_foreach: reading file "/home/local_user/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/local_user/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from [devshell-vm-xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.cloudshell.dev]:6000
debug3: order_hostkeyalgs: prefer hostkeyalgs: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: hmac-md5,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: hmac-md5,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ssh-rsa SHA256:oaU6VCSAN/xtJF6bMyDpuffYo6Cqsqsv44JsJ5Z/5/4
debug3: put_host_port: [xx.xx.xx.xx]:6000
debug3: put_host_port: [devshell-vm-xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.cloudshell.dev]:6000
debug3: hostkeys_foreach: reading file "/home/local_user/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/local_user/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from [devshell-vm-xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.cloudshell.dev]:6000
debug3: hostkeys_foreach: reading file "/home/local_user/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/local_user/.ssh/known_hosts:2
debug3: load_hostkeys: loaded 1 keys from [xx.xx.xx.xx]:6000
debug1: Host '[devshell-vm-xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.cloudshell.dev]:6000' is known and matches the RSA host key.
debug1: Found key in /home/local_user/.ssh/known_hosts:1
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: .ssh/id_rsa (0x56066a887910), explicit
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: .ssh/id_rsa
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (publickey).

EDIT: Result of sudo grep -i ssh /var/log/auth.log run from within my cloud shell:

sshd[569]: rexec line 33: Deprecated option KeyRegenerationInterval
sshd[569]: rexec line 34: Deprecated option ServerKeyBits
sshd[569]: rexec line 45: Deprecated option RSAAuthentication
sshd[569]: rexec line 52: Deprecated option RhostsRSAAuthentication
sshd[569]: reprocess config line 45: Deprecated option RSAAuthentication sshd[569]: reprocess config line 52: Deprecated option RhostsRSAAuthentication sshd[569]: error: Received disconnect from xx.xx.xx.xx port 54590:14: No supported authentication methods available [preauth]
sshd[569]: Disconnected from xx.xx.xx.xx port 54590 [preauth
  • I have verified that the private key is not corrupted
  • I have verified that the user name I am connecting with matches that which cloud shell is expecting
  • I have verified that the authorized_keys file on the Cloud Shell system has permissions "rw-r-r" (result is -rw-r--r-- 1 root root 2584 Aug 26 12:08 /etc/ssh/keys/authorized_keys)
Jason Dark
  • 323
  • 4
  • 10

1 Answers1

1

[UPDATE]

Run these commands which use the alpha SDK to generate/install an SSH key:

gcloud components install alpha
gcloud alpha cloud-shell ssh --dry-run

Then you can use the SSH key file ~/.ssh/google_compute_engine.

If this works, then your SSH key is the problem.

[END UPDATE]

In your debug output, line 87 and 88:

debug3: send packet: type 50
debug3: receive packet: type 51

Line 87 means "user auth request". Line 88 means "user auth failure".

The SSH server has rejected your SSH key.

Run this command in Cloud Shell to see the log for SSHD. You should see the exact error why your SSH key was rejected:

sudo grep -i ssh /var/log/auth.log

Possible problems/solutions:

  • Corrupted Private Key on your local system.
  • Private Key does not match the remote system Public Key in authorized_keys.
  • The user name "user" does not match the remote system.
  • Incorrect file permissions on remote system authorized_keys ( should be 0644 - "rw-r-r")
  • OpenSSH server cannot read authorized_keys (missing read file permission).

Note: Make sure that your private key (.ssh/id_rsa) is only readable by you (no write permissions for anyone, no read permissions for anyone else but you) - 0400.

John Hanley
  • 74,467
  • 6
  • 95
  • 159
  • Thanks for the suggestions: These are the logs generated when I try to connect: – Jason Dark Aug 25 '19 at 23:22
  • sshd[569]: rexec line 33: Deprecated option KeyRegenerationInterval sshd[569]: rexec line 34: Deprecated option ServerKeyBits sshd[569]: rexec line 45: Deprecated option RSAAuthentication sshd[569]: rexec line 52: Deprecated option RhostsRSAAuthentication sshd[569]: reprocess config line 45: Deprecated option RSAAuthentication sshd[569]: reprocess config line 52: Deprecated option RhostsRSAAuthentication sshd[569]: error: Received disconnect from xx.xx.xx.xx port 54590:14: No supported authentication methods available [preauth] hd[569]: Disconnected from xx.xx.xx.xx port 54590 [preauth] – Jason Dark Aug 25 '19 at 23:23
  • 1) Please do not post details as comments. Few will try to read them. Edit your question, format the new information so that we can read if easily, then post a comment that the question was updated. 2) I gave you a list of possible problems. Post the details after checking each possible problem. 3) Double-check the log file. Is there a line mentioning publickey? 4) If there is no error message in Cloud Shell's log file, your problem is probably the directory `~/.ssh` has the wrong file permissions or `~/.ssh/id_rsa` has the wrong file permissions. – John Hanley Aug 25 '19 at 23:32
  • Thanks for the pointers I have updated the main question. In my connection logs I can see debug1: key_load_public: No such file or directory which makes me think this could be a file permissions problem. Will look into it and update. – Jason Dark Aug 26 '19 at 00:26
  • 1) key_load_public - that is just a warning that the public key file does not exist. This is not required to authenticate as the private key will be used. 2) I doubt Cloud Shell has a user named user. The user name is determined by Google Cloud IAM identities (member accounts) unless you are just masking the real user name. 3) Launch Cloud Shell. In the terminal window execute `gcloud auth list`. That is your user identity. – John Hanley Aug 26 '19 at 00:51
  • I am not connecting with the username "user", I am using the correct username. I was just obfuscating this for my public post. In `gcloud auth list` I can see my email address as the user. – Jason Dark Aug 26 '19 at 01:27
  • If you understand the Go language, I wrote a program that connects to Cloud Shell supporting both SSH and SCP. This will show you the details of connecting to Cloud Shell. The code also shows how to launch the program `ssh` connecting to Cloud Shell with the correct parameters, keys, etc. https://github.com/jhanley-com/google-cloud-shell-cli-go – John Hanley Aug 26 '19 at 01:31