13

I built a web application that runs on a virtual server for one of my customers. It includes the facility for them to upload files, but they now want to store those files on a server in their office.

The easiest way to do this would be to mount their file system over the internet directly onto the virtual server. I have experience of mounting NFS over an internal network, but I am not clear if it would work in this scenario.

Any thoughts? I can always write new software to transfer the files, but this would be an easy fix!

Note: The server in the office is Ubuntu 12 server running on a virtual server in a windows hyper-v environment. The VPS with the web app is Ubuntu 11.04

Ben Holness
  • 944
  • 2
  • 10
  • 28

5 Answers5

18

No one has mentioned sshfs yet. If you're on a modern linux distro and have ssh access to the remote host, it's as simple as:

sshfs user@hostname:/remote/directory /local/directory

Performance is quite acceptable (but not nearly as fast as a streamed sync like rsync if you require the whole directory).

chicks
  • 3,793
  • 10
  • 27
  • 36
Andy Ross
  • 296
  • 1
  • 3
  • 1
    This seems like the best solution so far, the FAQ even says "Resources can be mounted over slow and unreliable (distant) connections." Another webpage tells me that "sshfs -o reconnect server:/path/to/mount" will have it automatically reconnect if the connection drops. –  Aug 17 '12 at 15:55
4

NFS is inherently insecure. It would be a very poor choice for connecting over the internet.

I like the post that mentions rsync. Instead of using cron to fire the transfer, I would hope that you could simply run the rsync job from your code that handles the file upload.

When the upload completes, rsync the file to their server, done.

You would need to set up a secure connection to their server for the transfer, I would expect.

If you wanted to, you could put the incoming files into a list for transfer, removing the names after successful copy, and give yourself some fail over capability, in case something happens to the connection.

As someone else already pointed out, rsync is designed to handle groups of files, or a hierarchy (thanks, spell check) , so this wouldn't be that hard to accomplish.

  • 1
    rsync has no problems using SSH as transport. – Ignacio Vazquez-Abrams Aug 17 '12 at 03:44
  • The web app already saves the files on the local server and references them there. There will be no space on the local server, so they will have to be removed once transferred, which means writing new code, hence I am looking for a mounted solution first, if possible. –  Aug 17 '12 at 04:05
2

NFS may be inherently insecure but that is not the fault of the service. Telnet and FTP are also inherently insecure but both have been used for decades on the open Internet. If an encrypted VPN tunnel is established, then the lack of NFS encryption is irrelevant.

Also if a firewall is configured to only allow a specific remote host address to connect to the NFS mountpoint, then a bare NFS connection is mostly secure from hacking, except from Three Letter Agencies that sniff Internet traffic, and which have access to powerful decryption to crack your encrypted remote connections anyway.

I see that some versions of NFS use UDP by default, which is likely where the Internet reliability problems come from. UDP packets can get lost if the connection is congested, and no retransmission is automatically attempted. If you want a reliable connection, make sure your NFS is using (or is forced to use only) TCP packets over the Internet.

Dale Mahalko
  • 755
  • 1
  • 6
  • 17
  • 1
    > Telnet and FTP are also inherently insecure but both have been used for decades on the open Internet Although true, I don't think this is a good point, because generally it is considered a (very) bad practice to use these over the internet now. Mozilla even suggests [deprecating unencrypted http](https://blog.mozilla.org/security/2015/04/30/deprecating-non-secure-http/). – Ian D. Scott Feb 23 '18 at 23:55
0

NFS can be done through a firewall. If you search, you will find others who have attempted and been successful with this process, however, it is not as straightforward as opening ports and appears to be dependent on the version of NFS being run.

rsync is an excellent suggestion and has a lot of flexibility. However, this would be a process run outside of your app. rsync can handle syncing of local file updates and maintain an entire directory in sync with other folders.

ssh would be another secure option and would only require a single port to be opened. I have used this in the past between Windows and Linux systems and has worked out well both from within an app as well as scheduling the process.

I'm sure there are other ways to accomplish what you are looking for, but we'd need more information about the types of files these are and if you are linking them into a page or streaming them in with something more complex.

KodeTitan
  • 881
  • 2
  • 10
  • 15
  • I understand the alternatives, mounting the filesystem would be the easiest solution, I just don't know how stable NFS is over the internet as opposed to over an internal network. What happens if the connection drops etc., or if there is a newer alternative designed to be robust to this. –  Aug 17 '12 at 03:39
0

NFS mounts can be done using automount, which will continue to try the mount on failure, but the only way I would consider using it over the pipes is with a vpn or some other secure connection (actually, I wouldn't consider it, but you sound fixed on the idea).

Even using automount, if there is a network issue, and you are putting their uploads on the NFS mount, the upload will likely fail or be corrupted, unless you are storing the file locally and then copying on upload success.