Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [distributed-filesystems]

128 questions
3
votes
1 answer

GlusterFS vs Ceph, which is better for production use in 2012?

This is the same question that was asked here, but it's been almost two years since. Meanwhile Ceph has seen constant development (361 kernel commits) and btrfs, in my opinion, is on the verge of production readiness. Both projects' web sites have…
al.
  • 925
  • 6
  • 17
3
votes
1 answer

Setting up a Rails 2.3.x app on EC2 for easy scalability

I'm running a simple rails stack on a single dedicated machine. We're reaching our full capacity and have absolutely no setup for scaling, just one app on one machine. I did some research and came up with a potential stack for scalability. I'm no…
Max Chernyak
  • 650
  • 8
  • 21
3
votes
2 answers

Distributed, Parallel, Fault-tolerant File System with high throughput

I am looking for DFS (distributed file system) that is fault tolerant and easy to maintain. I will have tons (100M+) of small files (from 1K to 500K). Files will be located in some directories what will build a logical structure of the data. I will…
Worker
  • 647
  • 1
  • 6
  • 9
3
votes
2 answers

How backup a distributed file system?

Note: This is a "theoretical" question, as I haven't got that kind of data yet. If you have a distributed file system spanning a dozen or more servers, and TBs of data, how do you perform backups of that? Local tape drives aren't an option as I am…
monster
  • 618
  • 3
  • 10
  • 17
3
votes
4 answers

Server setup for image storage

I need to store 25M Photos in 4 sizes = total 100M Files, the filesize will vary between 3Kb and 200 kb per file and the used storage at beginning is about 14-15 TB. Our goal is to have the data on 2-4 Server available and to serve them with a local…
Nenad
  • 375
  • 1
  • 5
  • 14
2
votes
1 answer

Which distributed filesystem will actually fit my needs?

Hear me out I have seen the question asked (in different forms) here, here, and perhaps the best one I found was here, but I do not think this is a duplicate because quite some time has past since those questions were asked, and my question has its…
Michael
  • 123
  • 4
2
votes
1 answer

How to make GlusterFS mount point highly available?

It's true that one can mount a GlusterFS volume on any brick and the FUSE client will automatically discover the other bricks and connect to them as well. If the initial brick fails the client will fail over to one of the other bricks. So, after the…
Martijn Heemels
  • 7,728
  • 7
  • 40
  • 64
2
votes
1 answer

Glusterd dead but cluster still running

We are currently running a three node cluster, on Gluster 3.6.4. On one of our nodes we noticed that the glusterd daemon is dead. But the glusterfsd daemons are still running, and we believe clients are connecting and retrieving data We noticed that…
Vorsprung
  • 183
  • 6
2
votes
1 answer

distributed storage solutions

I've got some desktop PCs available in my company since the last user hardware renewal and I would like to re-use them for a demo plaform. They are powerfull enought to emulate a few servers. I guess I could do some virtualization farm, however…
elge
  • 121
  • 2
2
votes
3 answers

LizardFS/MooseFS -- Multiple Storage Volumes

I'm using the new LizardFS and it's awesome! I'm new to LizardFS/MooseFS overall, but it seems pretty straight-forward. I was wondering two things: First, is there a way to setup separate storage pools? For instance, I could have five servers each…
Nick Coons
  • 387
  • 4
  • 16
2
votes
4 answers

Scale-out distributed storage with snapshots

I'm aware there are many similar questions around, several with great answers. I still haven't quite found what I'm looking for though: a distributed, scale-out FS that supports snapshots. Gluster with finished snapshot support would be great, but…
user206444
2
votes
1 answer

Fault-tolerant network access to redundant storage in a heterogeneous environment (including Windows)?

We are facing a design "challenge" where we are required to set up a storage solution with the following properties: What we need HA a scalable storage backend offline/disconnected operation on the client to account for network…
Adrian Frühwirth
  • 376
  • 5
  • 16
2
votes
0 answers

How do I speed up “ls -l” on a network file system (NFS)?

I have a NFS volume mounted in the following way: theHostname.com:/path/on/theHostname on /local/path type nfs (rw,nosuid,addr=10.0.0.54) The /local/path directory has 30_000 files. When I do a ls in /local/path it takes about 1s on average. When I…
Wojtek
  • 465
  • 1
  • 4
  • 6
2
votes
0 answers

NFS showing files in directory, but can't open or stat

I'm using a network of Linux (Debian Squeeze on kernel 2.6.32) machines, sharing files using NFS (v3). The scenario is that a process running on client A will create a file through NFS on file server Z. Then after the process is finished running on…
user79126
  • 449
  • 1
  • 4
  • 9
2
votes
1 answer

distributing access across multiple disks

I have a rather large application that requires simultaneous access to rather a lot (300+) of large files, and accesses them by reading them all at the same time. If all the files are put on 1 disk (if I have a disk that's even big enough), the…
camelccc
  • 255
  • 1
  • 15
1 2 3
…
8 9