I have a website that offers FTP space. It's a secondary function to the main site, but a feature that is proving popular. I use ProFTPd with MySQL which nicely sets the users up in their home directories and locks them in etc. It also give user FTPs access, for security.
The trouble I have now, is that of scaling.
I'm fast running out of space on my 4tb server and need to offload the FTP traffic onto multiple servers. However, I seem to have hit a stumbling block with every scenario.
Scenario 1: I just run multiple servers and assign individual users FTP space on each server. Advantages include the traffic not bottlenecking on the main server, however it doesn't allow user A (for example) to upgrade to 500gb of storage if there is only 100gb left on the drive. (It may happen).
Scenario 2: Run something like GlusterFS and create a pool of servers as a distributed file system and mount my users directory on the main server as a Gluster client. Disadvantage as I see that is that if I have 50 FTP servers in the future (maybe), then all 50 servers are going to be pointing traffic at my one main server and the bottleneck will be tremendous.
Scenario 3: Use something like S3FS and just mount a bucket (or several). Advantages being infinity scalable, but disadvantages include above bottlenecks and phenomenal bandwidth bills from AWS.
Could anybody suggest a "best route" for the scenario I describe. Forget existing clients, I am happy to move data and directory entries in the DB one at a time if it means I can get a working model. I just didn't realise it would become popular so didn't give it a great deal of advance thought.
So ideally, what I'm looking to do is have the ability to create FTP users, with quotas, on the fly from PHP as I do now, create a home directory and lock users into it, but have the ability to add extra servers (with redundancy) as required that will scale well and not bottleneck. And if SFTP over SSH were possible to add to all that, it would be a heavenly result.
I can do all of the things above individually, but cannot seem to find a solution that fits my needs to cover all of them.
Your help and advice would be greatly appreciated.
