Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [distributed-filesystems]

128 questions
1
vote
3 answers

Can I use RAM as Cache for a secondary storage device

I have 10TB~ RAID 5 array and two NVME 500GB on RAID 0 and they in different systems. what I want is to make a stick of RAM like 16GB or something to act as a cache or buffer to give me faster access time through the network to the second…
Raygo
  • 11
  • 2
1
vote
0 answers

Heavy network usage withg GlusterFS FUSE client

I have a simple GlusterFS 7.2 setup with 3 servers: data1 and data2 contribute several bricks (4 each) to a distributed replicated GlusterFS volume. host1 uses the GlusterFS FUSE client to mount the shared filesystem, but does not contribute…
Ben
  • 111
  • 1
1
vote
1 answer

Ridiculous Number of Files in a Directory

My employer acquired a company with one particular piece of software that stores a lot of PDF and PNG file files in one directory. When I first replicated it from AWS, there were about 11.5 million files. Now the number is approaching 13 millon…
scarville
  • 51
  • 6
1
vote
1 answer

Can kubernetes cluster nodes be geologically spread out?

Me and a couple of friends are thinking of setting up a kubernetes cluster where our homeservers will act as the nodes. As our nodes will be spread out between our apartments I am worried this will create problems when it comes to: Exposing…
Hannes Knutsson
  • 37
  • 1
  • 3
1
vote
1 answer

AWS EFS - file delete and recreate not detected programmatically for 25 to 30 seconds

I am observing a very large delay in EFS detecting that a file has been re-created i.e. deleted and then created again. In this simple test example, I have a single file that gets deleted and recreated around 5 seconds later. I have two EC2…
Ben
  • 121
  • 5
1
vote
1 answer

Multiple disks acting as one filesystem, while still allowing each disk to be accessed alone

I have a fairly large dataset (~160TB) that need to be delivered to a client every so often. This dataset consists of fairly large files, usually between 2Gb and 20Gb each. They exist on a BeeGFS filesisystem running on a RAID cluster with a total…
Jarmund
  • 535
  • 2
  • 6
  • 17
1
vote
2 answers

Does a distributed filesystem have to consist of multiple filesystems located on different computers?

Does the concept of a distributed filesystem require a distributed filesystem to consist of multiple filesystems located on different computers? Does an NFS filesystem consist of only one filesystem? (See the diagram below) If yes, is an NFS…
Tim
  • 1,487
  • 6
  • 28
  • 43
1
vote
2 answers

Do distributed file systems partition a file across nodes?

In HDFS, a file is partitioned into blocks which are distributed across several nodes. I am wondering if that is also true in the following distributed file systems: NFS, Andrew File systems (and Bayou, Coda, if you happen to know)? Thanks.
Tim
  • 1,487
  • 6
  • 28
  • 43
0
votes
0 answers

Will DFRS support a lot of data over a WAN?

I have a server in our home location that houses many of our engineers. This location has Comcast fiber 100 up and down. Internally, the network here is much faster than that. This location has Server 2016 Datacenter, and a lot of extra power. I…
GeoffM
  • 75
  • 8
0
votes
1 answer

glusterfs client mount point fails with 'Client quorum is not met'

Assume a basic replica 3 arbiter 1 configuration, glusterfs-server 4.0.2 and glusterfs-client 4.0.2. glusterfs-client is installed on Ubuntu 18.04. In order to verify that write / read ops are permitted when one storage node is down as stated in…
e2l3n
  • 101
  • 2
0
votes
2 answers

Networked In-Memory FileSystem w/ zram + XFS + GlusterFS - Best Practices

A few months ago I used XFS formatted zram devices strung together with GlusterFS to create a distributed / networked / replicated in-memory filesystem on a few bare metal servers (running RHEL 7.2). I'm using this FS as a performant way to store,…
Victor
  • 1
  • 2
0
votes
1 answer

GlusterFS server logic

I quite don't understand about GlusterFS probes. See, i add all servers to /etc/hosts file and i do this for each server. Lets say i have 100 servers and later i added 100 more. I cannot imagine updating all 200 server's hosts file for see each…
xangr
  • 144
  • 4
0
votes
1 answer

New files on Linux Samba share: Now you don't see me, now you do

When a file is uploaded to our server, we have to do some work on it to allow or reject it. Some files need to be opened in a Windows box (OpenXml Sdk performance reasons). Our web server runs in a Linux box. There is also a samba share in Linux. As…
Nomada
  • 101
  • 2
0
votes
1 answer

Expanding/Growing OCFS2 forever

I have a kvm environment where a shared disk mounted by 2 VMs through rbd ceph. What I would like to know that in the future how is it possible to expand OCFS2 filesystem what I planning to use on top of it: LVM2 is not an option: "As of this…
pluto73
  • 29
  • 1
  • 4
0
votes
1 answer

Distributed storage with UNC share?

What approach would you take to build a massive storage, using multiple PC's running Linux OS? The recipe I would like to use sounds like this: take 10 PCs equipped with six 2TB disks, install your favorite flavor of *NIX OS on each PC, configure…
user11925
  • 51
  • 1
  • 3
1 2 3
…
8 9