permission denied on authorized_keys

Question

I am attempting to set up SFTP for chrooted users, and use SSH public key authentication. In this example I'll be working with the dummy user "globocorp" who's a member of "sftpusers". This user is chrooted to /sftp/globocorp

I have placed my public key in the location specified in my sshd_config: /sftp/globocorp/sftpdirectory/.ssh/authorized_keys

When the remote user attempts to connect to the server via commandline SFTP, this message is logged on the server side:

debug1: trying public key file /sftpdirectory/.ssh/authorized_keys
debug1: Could not open authorized keys '/sftpdirectory/globocorp/.ssh/authorized_keys': Permission denied

I have reviewed the permissions and recommendations- executed these commands:

chown globocorp:sftpusers /sftpdirectory/globocorp/.ssh
chmod 700 /sftpdirectory/globocorp/.ssh
chmod 600 /sftpdirectory/globocorp/.ssh/authorized_keys

The ls -l output on the .ssh folder looks like:

drwx------ 2 globocorp         sftpusers 4.0K Nov  3 15:04 .ssh

and the individual file:

-rw------- 1 globocorp sftpusers  406 Nov  3 12:13 authorized_keys

Full debug info from sshd(Server side) follows:

debug1: sshd version OpenSSH_5.3p1
debug1: private host key: #0 type 0 RSA1
debug1: read PEM private key done: type RSA
debug1: private host key: #1 type 1 RSA
debug1: read PEM private key done: type DSA
debug1: private host key: #2 type 2 DSA
debug1: rexec_argv[0]='/usr/sbin/sshd'
debug1: rexec_argv[1]='-d'
Set /proc/self/oom_score_adj from 0 to -1000
debug1: Bind to port 22 on 0.0.0.0.
Server listening on 0.0.0.0 port 22.
debug1: Bind to port 22 on ::.
Server listening on :: port 22.
Generating 1024 bit RSA key.
RSA key generation complete.
debug1: Server will not fork when running in debugging mode.
debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8
debug1: inetd sockets after dupping: 3, 3
Connection from 192.168.102.109 port 38946
debug1: Client protocol version 2.0; client software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-1.99-OpenSSH_5.3
debug1: permanently_set_uid: 74/74
debug1: list_hostkey_types: ssh-rsa,ssh-dss
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST received
debug1: SSH2_MSG_KEX_DH_GEX_GROUP sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_INIT
debug1: SSH2_MSG_KEX_DH_GEX_REPLY sent
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: KEX done
debug1: userauth-request for user globocorp service ssh-connection method none
debug1: attempt 0 failures 0
debug1: user globocorp matched 'User globocorp' at line 150
debug1: user globocorp matched group list sftpusers at line 158
debug1: PAM: initializing for "globocorp"
debug1: PAM: setting PAM_RHOST to "192.168.102.109"
debug1: PAM: setting PAM_TTY to "ssh"
debug1: userauth-request for user globocorp service ssh-connection method     publickey
debug1: attempt 1 failures 0
debug1: test whether pkalg/pkblob are acceptable
debug1: temporarily_use_uid: 559/506 (e=0/0)
debug1: trying public key file     /sftp/globocorp/sftpdirectory/.ssh/authorized_keys
debug1: Could not open authorized keys '/sftp/globocorp/sftpdirectory/.ssh/authorized_keys': Permission denied
debug1: restore_uid: 0/0
debug1: temporarily_use_uid: 559/506 (e=0/0)
debug1: trying public key file /sftp/globocorp/sftpdirectory/.ssh/authorized_keys
debug1: Could not open authorized keys '/sftp/globocorp/sftpdirectory/.ssh/authorized_keys': Permission denied
debug1: restore_uid: 0/0
Failed publickey for globocorp from 192.168.1.19 port 38946 ssh2
Connection closed by 192.168.1.19
debug1: do_cleanup
debug1: do_cleanup
debug1: PAM: cleanup

Background info:
SELinux is disabled.
CentOS 6.5
Running OpenSSH_5.3p1
SFTP -vv output just says: "Permission denied (publickey,gssapi-keyex,gssapi-with-mic). Couldn't read packet: Connection reset by peer"

Answer 1

I got it!

Following the directions on this site: http://sysadmin.circularvale.com/server-config/rsa-authentication-with-chrooted-sftp-authorized_keys-location/

As root, I created a separate folder in:

 /usr/local/share/keys/globocorp/.ssh/

This folder is owned by root, permissions set to "755"
authorized_keys file is in this folder, and owned by the user, permissions set to 600.

sshd_config contains this line:

AuthorizedKeysFile /usr/local/share/keys/%u/.ssh/authorized_keys 

And this match block:

Match user globocorp
        ChrootDirectory /sftp/%u
        X11Forwarding no
        AllowTcpForwarding no
        ForceCommand internal-sftp -l VERBOSE -f LOCAL6
        PubkeyAuthentication yes
        PasswordAuthentication yes

So in conclusion:
The authorized_keys of a chrooted user CAN be in a location that the user is chrooted out of. This is because the chroot is not processed until after login. Permissions should be EXACTLY as described above- no other permissions worked. (Not 700 on the parent directory) Paths defined in sshd_config are absolute (/ = the / of the server, not the user's chroot!)

To debug this, I used this command to run sshd on a separate port(23) and not interrupt existing sessions:

/usr/sbin/sshd -d -p 23

And then attempted to connect via SFTP from a remote server. This caused the server side to output useful debug messages that explained clearly what was happening on my login attempts.

Answer 2

debug1: Could not open authorized keys '/sftp/globocorp/sftpdirectory/.ssh/authorized_keys': Permission denied

Check that the user with UID/GID 559/506 has at least traverse (execute) permissions for all of the directories in the path /sftp/globocorp/sftpdirectory/. If they don't then add it.