0

I have an issue using GlusterFS to replace an existing, non HA, NFS setup to share data between machines. A bit about the configuration first:

The proof-of-concept glusterfs is a setup with 2 machines and a replicated volume. Then there are 2 clients that import this volume. One client is an SFTP server for customers to connect and send their files and the other is the server where our applications runs.

The issue I'm ecountering is about users and permissions. The user that connects to the SFTP server will be jailed in a specific directory (chroot) and SFTP requires strict permissions (the directory to be owned by the root user and root to be the only user allowed write permission) At the same time the application, running as a specific user, on the other server will need full access on the directory tree.

The setup I'm replacing uses NFS and the export is mounted with different ownership/permissions (using the uid and gid option while mounting) on the two clients; this way the users on the two servers have the permissions they need.

GlusterFS, on the other hand, AFAIK, doesn't allow a volume to be mounted with specific ownership on each machine. I'm aware GlusterFS is POSIX compliant and I can make use of the standard permissions systems and ACL

I've thought and/or tested few options, but none is satisfactory to me.

  • Using ACL: when adding the app server's user the read/write permission, it will consider that permission a "group" permission and SFTP will complain.

  • Creating a common user across the machines: not very flexible, relies on sysadmins to mantain the common user, and when moving to production the situation will complicate because more systems will have to interact.

  • Mount on a different place and bind to the correct directory. But then I discovered I can't change ownership...

  • Using NFS/Ganesha or SAMBA. This is overkilling, I can install the glusterfs client and I don't want to use additional layers that have to be configured and mantained.

The volume itself and all its content are owned by user root and group root.

Does any of you have a better idea? Or knows a feature of GlusterFS that allows me a simpler setup?

Thanks in advance.

ColOfAbRiX
  • 1,080
  • 2
  • 12
  • 23

1 Answers1

1

I think I've found an acceptable solution. I forgot about a behaviour of the SFTP daemon. It's true that SFTP need the jail directory to be owned by user root and only root to have write permissions. But the subdirectories can have any permission.

Instead of configuring ACL to the root of the gluster volume, I change the ACL only for the content of a jail directory, in this way:

Client 1 - App Server

root@appsrv$ cd /path/to/gluster/volume
root@appsrv$ chown -R root:root *
root@appsrv$ chmod -R 2750 *
root@appsrv$ ls -la
total 6
drwxr-x--- 14 root    root      ./
drwxr-xr-x  4 appuser appgroup  ../
drwxr-xr-x  3 root    root      .trashcan/
drwxr-s---  5 root    root      User1/
drwxr-s---  5 root    root      User2/
drwxr-s---  5 root    root      User3/

root@appsrv$ setfacl -Rm u:appuser:rwx */*
root@appsrv$ setfacl -Rm g:appgroup:rx */*
root@appsrv$ setfacl -Rdm u:appuser:rwx */*
root@appsrv$ setfacl -Rdm g:appgroup:rx */*

root@appsrv$ ls -la
total 6
drwxr-x---  14 root    root      ./
drwxr-xr-x   4 appuser appgroup  ../
drwxr-xr-x   3 root    root      .trashcan/
drwxr-s---+  5 root    root      User1/
drwxr-s---+  5 root    root      User2/
drwxr-s---+  5 root    root      User3/

Client 2 - SFTP Server

root@sftpsrv$ cd /path/to/gluster/volume
root@sftpsrv$ setfacl -Rm g:sftpgroup:rx *
root@sftpsrv$ setfacl -Rm g:sftpgroup:rwx */input
root@sftpsrv$ setfacl -Rdm g:sftpgroup:rx *
root@sftpsrv$ setfacl -dRm g:sftpgroup:rwx */input

root@sftpsrv$ ls -la
total 6
drwxr-x---  14 root root ./
drwxr-xr-x   4 root root ../
drwxr-xr-x   3 root root .trashcan/
drwxr-x---+  5 root root User1/
drwxr-x---+  5 root root User2/
drwxr-x---+  5 root root User3/

Now the SFTP works correctly and doesn't complain about permissions and the application server has full access where it needs

ColOfAbRiX
  • 1,080
  • 2
  • 12
  • 23