Distributed filesystem for storing temporary files

Question

We have a cluster of web servers that currently stores temporary files (that have been uploaded by users) on a central shared filesystem. The limitation being that anytime the server containing the shared filesystem goes down our entire site is down (or crippled at least). I've looked at solutions like Gluster and GridFS to create a fault tolerant filesystem but none seemed ideal for ephemeral files that are frequently being created and deleted. Security is also important and Gluster didn't seem to have any means of encryption over the wire.

What are some good fault-tolerant distributed filesystem options that have the capability of being secure and handling frequent file creation / deletion?

score 1 · Answer 1 · answered Oct 15 '13 at 23:59

1

openAFS --- (open Andrew File System)

answered Oct 15 '13 at 23:59

Michael Martinez

2,645
3
24
35

2

While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. – Ward - Trying Codidact Oct 16 '13 at 02:25
1

@Ward: There are no hyperlinks in my answer. openAFS is a distributed file system that does exactly what the OP needs. – Michael Martinez Oct 16 '13 at 17:33

jerm · Answer 2 · 2013-10-16T16:56:27.833

1

Depends on how want/need to access the filesystem. If you don't mind using API calls you could do something as simple as S3, or to keep it in-house run a RIAK cluster for a similar experience.

If that's too heavyweight you could run something like etcd (https://github.com/coreos/etcd). etcd is designed to run on every node and it's very high performance, if a bit more work to interface with.

edited Oct 16 '13 at 16:56

answered Oct 16 '13 at 03:14

jerm

637
3
5

RIAK's S3 clone looks very intriguing. Thanks! – Dan Oct 22 '13 at 20:52

score 1 · Accepted Answer · answered Oct 16 '13 at 17:03

Warning: opinionated content.

Distributed filesystems suck. There are good distributed object stores like cassandra and ceph, but invariably any filesystem emulation on top is a problematic beast, because filesystem operations don't map very well to distribution (locking, fsync()ing), and that leaves out the issue of needing to put code in the kernel to support them instead of in your application.

I do appreciate the simplicity of making all your content accessible via the filesystem, but maybe it's easier to write a bit more code and use a distributed object store like it's meant to be used.

Distributed filesystem for storing temporary files

3 Answers3