Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [filesystems]

A file system (or filesystem) is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device(s) which contain it.

A file system (or filesystem) is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device(s) which contain it. A file system organizes data in an efficient manner and is tuned to the specific characteristics of the device. A tight coupling usually exists between the operating system and the file system. Some file systems provide mechanisms to control access to the data and metadata. Ensuring reliability is a major responsibility of a file system. Some file systems allow multiple programs to update the same file at nearly the same time.

1986 questions
33
votes
4 answers

Is there such a thing as a 'quick' format for ext4?

In Windows, it's very fast to format with NTFS. I have a low powered Linux machine, with little RAM. Formatting a 2TB volume to ext4 takes a long long time. Is there anything I can do to speed up the format? I can't imagine what takes so long? …
stuck
  • 687
  • 2
  • 10
  • 23
33
votes
8 answers

How to workaround the NTFS Move/Copy design flaw?

As anyone that has dealt with file server permissions is aware, NTFS has an interesting design feature/flaw known as the Move/Copy problem. As explained in this MS KB article, the permissions for a folder or file do not automatically inherit from…
David Archer
  • 599
  • 1
  • 6
  • 16
32
votes
5 answers

What are the performance implications for millions of files in a modern file system?

Let's say we're using ext4 (with dir_index enabled) to host around 3M files (with an average of 750KB size) and we need to decide what folder scheme we're going to use. In the first solution, we apply a hash function to the file and use two levels…
Leandro Moreira
  • 549
  • 1
  • 7
  • 13
32
votes
6 answers

What are .DS_Store files and why does OSX leave them on Windows shares?

Whenever I access windows shares from OSX 10.5 it leaves .DS_Store files on the remote filesystem. What are they used for, and are they necessary, and can they be prevented from being created?
duckworth
  • 700
  • 3
  • 10
  • 12
32
votes
9 answers

Best way to compare (diff) a full directory structure?

What's the best way to compare directory structures? I have a backup utility which uses rsync. I want to tell the exact differences (in terms of file sizes and last-changed dates) between the source and the backup. Something like: Local file …
Adam Matan
  • 13,194
  • 19
  • 55
  • 75
31
votes
4 answers

Performance difference between ramfs and tmpfs

I need to setup an in memory storage system for around 10 GB of data, consisting of many 100 kb single files(images). There will be lots of reads and fairly periodic writes(adding new files, deleting some old ones). Now, I know that tmpfs behaves…
Ivan Kovacevic
  • 1,801
  • 3
  • 15
  • 19
31
votes
5 answers

Turning off atime on a filesystem

I am setting up a mongoDB replica set and one of the first things I am suppose to do is turn off atime on the file system. After researching this a bit, I am not opposed to doing this, but I have to ask, what uses atime? I have searched the…
QWade
  • 917
  • 1
  • 11
  • 17
31
votes
11 answers

Distributed Storage Filesystem - Which one/Is there a ready to use product?

With Hadoop and CouchDB all over in Blogs and related news what's a distributed-fault-tolerant storage (engine) that actually works. CouchDB doesn't actually have any distribution features built-in, to my knowledge the glue to automagically…
serverhorror
  • 6,478
  • 2
  • 25
  • 42
30
votes
2 answers

Transparent compression filesystem in conjunction with ext4

I am trying to test a project that needs compressed storage with use of the ext4 file system since the application I use relies on ext4 features. Are there any production/stable solutions out there for transparent compression on ext4? What I have…
user235918
  • 319
  • 1
  • 3
  • 7
30
votes
3 answers

The XFS filesystem is broken in RHEL/CentOS 6.x - What can I do about it?

Recent versions of RHEL/CentOS (EL6) brought some interesting changes to the XFS filesystem I've depended on heavily for over a decade. I spent part of last summer chasing down an XFS sparse file situation resulting from a poorly-documented kernel…
ewwhite
  • 197,159
  • 92
  • 443
  • 809
28
votes
6 answers

Preferred format of file names which include a timestamp

As we all know "unix" can have anything in a file except '/' and '\0', sysadmins however tend to have a much smaller preference, mainly due to nothing liking spaces as input ... and a bunch of things having a special meaning for ':' and '@' among…
James Antill
  • 751
  • 1
  • 5
  • 10
26
votes
5 answers

How do I identify the file system of a drive in Mac OS?

I have an external drive hooked up to my Mac, and I'm trying to determine things like, e.g., is this HFS or FAT, is it 32-bit or 64-bit, etc. It seems like there should be some trivial command that gives me this info, but I can't seem to find one.…
David Moles
  • 444
  • 1
  • 4
  • 13
26
votes
12 answers

Storing and backing up 10 million files on Linux

I run a website where about 10 million files (book covers) are stored in 3 levels of subdirectories, ranging [0-f]: 0/0/0/ 0/0/1/ ... f/f/f/ This leads to around 2400 files per directory, which is very fast when we need to retrieve one file. This…
BenMorel
  • 4,507
  • 10
  • 57
  • 85
25
votes
7 answers

Do I need to defrag unix filesystems?

I know Windows file systems are very sensitive to fragmentation and Unix such as UFS, ZFS, ext3, or ext4 are not as sensitive. But, is there any reason to defragment these filesystems at all? If so, what tools are available for that ?
paulgreg
  • 4,154
  • 6
  • 33
  • 32
25
votes
6 answers

What's the maxium number of files a Unix folder can hold?

What's the maxium number of files a Unix folder can hold? I think it will be the same as the number of files.
PERR0_HUNTER
1 2
3
…
99 100