Pubkey SSH fails with "we did not send a packet, disable method" in freebsd jail

Question

I have a FreeBSD VPS with 2 jails, each setup with ezjail (I know now that this is largely deprecated, but didn't at the time).

$ jls
   JID  IP Address      Hostname                      Path
     1  172.16.1.1      wwwserver                     /usr/jails/wwwserver
     2  172.16.1.2      wwwgit                        /usr/jails/wwwgit

The host and the jails are all running 12.2-RELEASE-p2.

I have key-based ssh login enabled in each jail, as well as the host. This works fine for the host and wwwserver, but not wwwgit. For that jail, I get this log:

debug1: Reading configuration data /Users/chris/.ssh/config
debug1: /Users/chris/.ssh/config line 3: Applying options for *
debug1: /Users/chris/.ssh/config line 22: Applying options for waitstaff_git
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 47: Applying options for *
debug2: resolve_canonicalize: hostname {censored-ip-address} is address
debug2: ssh_connect_direct
debug1: Connecting to {censored-ip-address} [{censored-ip-address}] port 22.
debug1: Connection established.
debug1: identity file /Users/chris/.ssh/id_ed25519_chrisdeluca_git type 3
debug1: identity file /Users/chris/.ssh/id_ed25519_chrisdeluca_git-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.9 FreeBSD-20200214
debug1: match: OpenSSH_7.9 FreeBSD-20200214 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to {censored-ip-address}:22 as 'git'
debug3: hostkeys_foreach: reading file "/Users/chris/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/chris/.ssh/known_hosts:7
debug3: load_hostkeys: loaded 1 keys from {censored-ip-address}
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:nhwOgcMl+Z+47Qu1VHAnjGnSbIdnjqMV60XQ9ilsCrI
debug3: hostkeys_foreach: reading file "/Users/chris/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /Users/chris/.ssh/known_hosts:7
debug3: load_hostkeys: loaded 1 keys from {censored-ip-address}
debug1: Host '{censored-ip-address}' is known and matches the ECDSA host key.
debug1: Found key in /Users/chris/.ssh/known_hosts:7
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /Users/chris/.ssh/id_ed25519_chrisdeluca_git ED25519 SHA256:xUYB2rlHSwtkA515PXWHC3dN8XQkcG2dbXJg1SPikxM explicit agent
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,keyboard-interactive
debug3: start over, passed a different list publickey,keyboard-interactive
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /Users/chris/.ssh/id_ed25519_chrisdeluca_git ED25519 SHA256:xUYB2rlHSwtkA515PXWHC3dN8XQkcG2dbXJg1SPikxM explicit agent
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,keyboard-interactive
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug3: send packet: type 50
debug2: we sent a keyboard-interactive packet, wait for reply
debug3: receive packet: type 60
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1
Password for git@waitstaff:

At first I thought maybe my permissions were off, but I can confirm I have the public keys uploaded to the git user's .ssh/authorized_keys file, and the permissions are correct:

drwx------  2 git  git  512 Dec 29 22:07 .ssh
-rw-------  1 git  git  109 Dec 29 22:13 authorized_keys

The SSH config itself is nearly identical across the host and jails.

Host

$ grep -E -v '^$|^#' /etc/ssh/sshd_config
Subsystem   sftp    /usr/libexec/sftp-server
PermitRootLogin without-password

wwwserver

$ sudo jexec wwwserver grep -E -v '^$|^#' /etc/ssh/sshd_config
Port 2222
AuthorizedKeysFile  .ssh/authorized_keys
ChallengeResponseAuthentication no

wwwgit

$ sudo jexec wwwgit grep -E -v '^$|^#' /etc/ssh/sshd_config
AuthorizedKeysFile  .ssh/authorized_keys
Subsystem   sftp    /usr/libexec/sftp-server

I also have a local ssh config file, which might be helpful. Here's the relevant contents.

IdentitiesOnly yes

Host *
  AddKeysToAgent yes
  UseKeychain yes

...

# Freebsd host
Host waitstaff
  Hostname {censored-ip-address}
  Port 22
  IdentityFile ~/.ssh/id_ed25519_waitstaff
  User freebsd

# wwwserver jail
Host waitstaff_deploy
  Hostname {censored-ip-address}
  Port 2222
  IdentityFile ~/.ssh/id_ed25519_waitstaff_deploy
  User chris

# wwwgit jail
Host waitstaff_git
  Hostname {censored-ip-address}
  IdentityFile ~/.ssh/id_ed25519_chrisdeluca_git
  User git

I'm at a loss about what's wrong. Any help figuring this out would be greatly appreciated. Thanks in advance!

Edit: In case it's pertinent, I changed the home directory for the git user (the user I'm trying to login as) to /git.

Check /var/log/auth.log in your `wwwgit` jail. Please note that your ssh client detects two identity files and the second one `/Users/chris/.ssh/id_ed25519_chrisdeluca_git-cert` (note -cert suffix here) is invalid or doesn't exist so you get `we did not send a packet, disable method`. For the first identity file, you get 'receive packet: type 51' which means auth failure. — AlexD, Feb 06 '21 at 09:00
Have you checked logs on server side? May be there is answer. — Alexander Tolkachev, Feb 07 '21 at 16:05

score 3 · Answer 1 · answered Feb 09 '22 at 13:51

I suggest checking OpenSSH version. I recently got such error after upgrading my client's OpenSSH version to 8.8. https://www.openssh.com/releasenotes.html

Incompatibility is more likely when connecting to older SSH implementations that have not been upgraded or have not closely tracked improvements in the SSH protocol. For these cases, it may be necessary to selectively re-enable RSA/SHA1 to allow connection and/or user authentication via the HostkeyAlgorithms and PubkeyAcceptedAlgorithms options. For example, the following stanza in ~/.ssh/config will enable RSA/SHA1 for host and user authentication for a single destination host:
Host old-host
    HostkeyAlgorithms +ssh-rsa
    PubkeyAcceptedAlgorithms +ssh-rsa

score 1 · Answer 2 · answered Feb 04 '21 at 18:39

It could be the permissions of the home directory of the user you're trying to log into which is not accepted by the ssh server, could you post those please?

If you want to keep the permissions as they are, you could change the sshd_config to include this to override the permission check:

StrictModes no

Explaination of StrictModes from man sshd_config:

 StrictModes
         Specifies whether sshd(8) should check file modes and ownership of the user's files and home directory before accepting login.  This is
         normally desirable because novices sometimes accidentally leave their directory or files world-writable.  The default is yes.  Note
         that this does not apply to ChrootDirectory, whose permissions and ownership are checked unconditionally.

That's a really good thought; unfortunately after disabling StrictModes I still get the "we did not send a packet, disable method" error :( — bronzehedwick, Feb 06 '21 at 01:42

Pubkey SSH fails with "we did not send a packet, disable method" in freebsd jail

2 Answers2