Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [distributed-filesystems]

128 questions
1
vote
1 answer

Storing Large Amount of files in a file system?

Guys I am creating an application which will be saving a .txt file (approx 10 KB/file I guess) in a file system and will be saving its path in MS SQL database. Now as per my estimation, approximately 350-400 million files of this nature will need to…
NewbieProgrammer
  • 119
  • 4
1
vote
2 answers

Setting up a multi-host SAN

I'm looking for some advice on setting-up multi-host I/O access on our SAN: I have a blade enclosure (PowerEdge1000e) containing an Equallogic PS-M4110 storage blade with a single RAID6 volume currently formatted as ext4. This is connected via iSCSI…
Eric S Nyberg
  • 11
  • 1
1
vote
2 answers

Simple, decentralized, automatically synced, local cached file share for linux server

I wonder if someone could recommend me some solution to sharing files between a small amount of linux VPS (virtual private servers). Basically I have a bunch of linux VPS to manage, and I want to share files between them. The typical use cases are…
icando
  • 229
  • 2
  • 8
1
vote
2 answers

Distributed storage and computing

Dear Serverfault community, After researching a number of distributed file systems for deployment in a production environment with the main purpose of performing both batch and real-time distributed computing I've identified the following list as…
Tim van Elteren
  • 11
  • 1
1
vote
3 answers

Distributed filesystem for storing temporary files

We have a cluster of web servers that currently stores temporary files (that have been uploaded by users) on a central shared filesystem. The limitation being that anytime the server containing the shared filesystem goes down our entire site is down…
Dan
  • 657
  • 2
  • 9
  • 13
1
vote
1 answer

DFS - Stop sync of large folder that has since been removed

We have a site to site DFSR on Windows Server 2008 R2 that has been running perfectly between site A to site B until someone dumped a 20GB folder. This has overwhelmed the upload and make the internet almost useless at site A (the upload is low at…
morleyc
  • 1,150
  • 13
  • 47
  • 89
1
vote
3 answers

Moving Away from NFS

I've been working on moving more and more services away from a reliance on NFS, and I'm wondering what others have done with this issue. I know there are distributed file systems out there, and I have some experience using one of them (mogilefs). …
gabe.
  • 268
  • 1
  • 6
1
vote
1 answer

Is it possible to set the cache size for fuse and/or sshfs?

I can't seem to find something in the documentations. Is it possible to set the cache size for fuse and/or sshfs and if so, how?
Jan Deinhard
  • 2,383
  • 5
  • 26
  • 33
1
vote
1 answer

How Does DFS Replication in Windows 2008 Handle Write/Delete in Rapid Succession

We'd like to replicate a mail drop folder between two servers; however, as files are written to the source folder, a timer job on that server is also regularly processing and deleting them, meaning they may not last very long. My question is: does…
Nariman
  • 229
  • 3
  • 10
1
vote
2 answers

GlusterFS with RAID-1

Is it possible to use GlusterFS to have more flexible FS and to support higher capacity + RAID-1 to have redundancy to avoid data loss in case of failure? I need to get a mirrored FS that has high flexibility in scaling (adding more HDs and/or…
Khaled
  • 36,533
  • 8
  • 72
  • 99
1
vote
3 answers

which is the best choice of serving static media files of a website with heavy traffic?

Take a look at this list http://en.wikipedia.org/wiki/List_of_file_systems#Distributed_parallel_fault-tolerant_file_systems and which is the best choice for storing large number of static media files (with normal file size: 10KB-2M for images, and…
Mickey Shine
  • 939
  • 4
  • 17
  • 33
1
vote
2 answers

How to store all static media files of a website with heavy traffic?

I heard of some distributed file systems like Gluster and Moose. Are there any better ideas out there that I am not aware of?
Mickey Shine
  • 939
  • 4
  • 17
  • 33
1
vote
1 answer

Is there a remote filesystem for Linux that's encrypted and resilient against unreliable servers?

I've tried many combinations of sshfs and SMB, with autofs and/or manual mounts in a Cron job, and I've always had freezing / long timeout issues when servers become unavailable or weren't available to begin with. My requirements are as…
user137063
  • 11
  • 1
1
vote
1 answer

Docker Swarm Distributed File System with local storage

I'm looking for a distributed file system solution/network file system which can be used in the following scenario: I have a Docker Swarm with many hosts, but each host is essentially self-contained and looks no different than any other host. We…
Alex Waldenmaier
  • 11
  • 2
1
vote
0 answers

What's the simplest way of "resetting" multiple linux clients to restore the default state of a fresh install?

We have 6 linux clients (mint) which are used for training. Courses might run anywhere from a few days to multiple weeks. They want a "one button press", as in very quick and easy, way of resetting all the clients back to their fresh install state…
zorban
  • 11
  • 1
1 2 3
…
8 9