Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [distributed-filesystems]

128 questions
4
votes
2 answers

Linux filesystem or CDN for millions of files with replication

Please tell me solution for this scenario: several millions files, located in one directory ("img/8898f6152a0ecd7997a68631768fb72e9ac2efe1_1.jpg") ~80k file size in average 90% random read access backup (replication) to other servers (every 5…
Roman Skvazh
  • 95
  • 8
4
votes
1 answer

Recommendations for distributed processing/distributed storage systems

At my organization we have a processing and storage system spread across two dozen linux machines that handles over a petabyte of data. The system right now is very ad-hoc; processing automation and data management is handled by a collection of…
Eddie
  • 323
  • 1
  • 2
  • 5
4
votes
2 answers

Using DFS roots as shared folders

I am curious if I can use DFS roots as shared folders without using any DFS links. Some background: I like the idea of using DFS for name abstraction. By using domain-based namespaces, I can abstract away server name reference and have the…
ejel
  • 173
  • 1
  • 4
  • 10
4
votes
3 answers

How can I tie togeather extra space on Macintosh desktops with a distributed filesystem?

I have access to a bunch of Mac desktops, the hard drives of which are under-utilized. I want to set up a distributed filesystem to gang them together into one large virtual volume. The server has to be able to run as a normal user. I've tried…
interfect
  • 313
  • 1
  • 2
  • 7
4
votes
1 answer

What are the functionalities of Distributed File systems and Distributed Storage Systems?

i'm reading cloud vendors solutions for the distributed storage systems such as Amazon Dynamo and Google Big Table. and really confused in two terms : what is Distrubuted file systems, Do cloud vendors use different kinds of DFS ? what is…
Berkay
  • 431
  • 4
  • 18
4
votes
1 answer

Distributed File Systems

So, I've been reading several articles around ServerFault as well as google. (For Example, this link) My Requirements are very similar to the link above, however i'd like to also have dynamic or at least resizeable file volumes, so if necessary i…
grufftech
  • 6,760
  • 4
  • 37
  • 37
3
votes
1 answer

Does Perkeep (camlistore) have built in protection from bitrot?

Does Perkeep (PKA Camlistore) offer protection from silent corruption (eg. bitrot) of the data in its current design like what is offered in ZFS. If it does, how well does it fare when compared to ZFS? I know that a Perkeep node can be run on ZFS…
Timothy C. Quinn
  • 299
  • 1
  • 2
  • 10
3
votes
3 answers

Choice of distributed filesystem for intensive reads and writes

I have a series of servers (HP ProLiant, 34 servers) each of which with 500 G of hard drive space. These servers are part of a computational cluster which runs processes that roughly fall into two "phases": phase 1: reading a small number of large…
Einar
  • 225
  • 2
  • 11
3
votes
0 answers

Are snapshots and clones filesystem-wide in ZFS-backed Lustre clusters

My goal is to find a distributed filesystem on Linux that supports ZFS-like lightweight snapshots and snapshot clones. This StackOverflow question expresses what I'm looking for pretty well. I'm trying to figure out whether Lustre + ZFS is the…
Anand
  • 31
  • 2
3
votes
3 answers

failsafe RAM drive solution

I'm looking for a production solution to create a RAM drive that will be safely synchronized with HDD. I have a piece of custom software with heavy I/O load (this is some kind of proprietary document-oriented DB), and I need to speed it up…
user1450663
  • 151
  • 6
3
votes
1 answer

Distributed filesystem for automated offline data mirroring

I'd like to achieve the following setup: Every time I connect my laptop to a local network, my partition gets automatically mirrored to a partition on my local server. I only want to mirror what has changed from the last time. (I understand that it…
Petr
  • 581
  • 1
  • 5
  • 16
3
votes
1 answer

Are there any serverless distributed filesystems?

Similar to Is there a Distributed SAN/Storage System out there? or Distributed redundant storage solution?, except with a couple key differences (and two years newer). Context: I have a small Rocks cluster (a few dozen machines of varying…
zebediah49
  • 176
  • 7
3
votes
2 answers

Umount stale glusterfs partition

I am using glusterfs on several Ubuntu servers: two of them are running glusterfs servers in replication mode. Without any clear error, the glusterfs partition became stale and the system shows this error when I try to access the stale…
Khaled
  • 36,533
  • 8
  • 72
  • 99
3
votes
1 answer

Is distributed storage on an internal network possible?

I've been thinking about all the workstations I have on my shop floor (about 50) and the wasted drive space each of them has. For instance, my machines only use about 30G to 40G of local storage, yet they're coming from the manufacturer with 500G…
Albion
  • 465
  • 2
  • 6
  • 16
3
votes
1 answer

How to install ceph on EC2 Amazon Linux AMI

I want to test Ceph (a distributed network storage and file system) on some EC2 hosts which is derived from Amazon Linux AMI (amzn-ami-2011.09.2.x86_64-ebs). The kernel version is 3.2 and btrfs is enabled. But kernel config options related to Ceph…
takaomag
  • 261
  • 1
  • 3
  • 7
1 2
3
…
8 9