3

I need a direction since all the blogs/benchs seems to contradict.

Goal:

Having a lot of small files on FS in a directory tree fashion similar to this AB/CD/EF... Two size groups : G1 : 10-20 kB G2 : 100-500kB

I need really fast access to server them with nginx.

Contraints

Delete or creation perf is not important. To an extrem, it could take a minute i would not care. Read/Access has to be fast <->nginx. Files are copied to different disks at their creation so no raid involved. Just commodity. Dynamic inode creation or similar. Reliability is not a major concern since there are duplicates on 3 different servers.

Context and contradictions

I read XFS suits high performances mail servers with many small files involved. Other say that XFS sucks at handling small files and EXT4 is better.

And it keeps contradicting all over Internet.

Problem

Overall the problem is inode allocation. Big inode tables waste space. Set it too weak and you are doomed: erase and recreate the partition (no way it can happen).

To me, it is a trap which can stops me from adding new files even if space is still available. That is why I wish to switch to something else.

And due to the highly dynamic nature and unpredictable number of files there, it seems that I am stuck. Hence the need for your opinion.

btrfs is considered not production ready. Others seems out (reliability)

What FS should I use in this case ?

Community
  • 1
Larry
  • 145
  • 1
  • 8

2 Answers2

0

ZFS and XFS allocate inodes dynamically so would fit the need. I wouldn't bet on ReiserFS.

jlliagre
  • 8,861
  • 18
  • 36
  • ReiserFS and XFS allocate inodes dynamically, too. – Vinz Dec 04 '13 at 09:20
  • Zfs seems neat. But for a new project, do you think it is backed up enough by Oracle in comparison with xfs ? What about the concurrent access ? I am on debian, and can tune the system to accept more than 1024 concurrent connection. Is it still possible there ? – Larry Dec 04 '13 at 10:03
  • Well, on Debian, that would be through zfsonlinux.org / open-zfs.org, which are not backed by Oracle but still by an active community. About performance, I guess nothing would replace experiments using your data and expected usage simulation. – jlliagre Dec 04 '13 at 10:47
  • 1
    ZFS on anything other than solaris is PHENOMONALLY slow. – symcbean Dec 04 '13 at 13:35
  • @symcbean or not ... I have used ZFS on Solaris since 2005 and the FUSE based implementation on Linux since 2008 and the latter was and is still very usable, not at all phenomenally slow. Even while I didn't test it yet, I expect the in kernel implementation to be much faster. – jlliagre Dec 04 '13 at 14:15
0

And it keeps contradicting all over Internet.

But you expect to get consistent answers here?

The only specific recommendation I would make is to forget about ZFS - unless you're running on Solaris (where it makes a lot of sense).

Beyond that, how you configure your disks, your I/O scheduler and having spare memory for disk I/O is much more important than the choice of FS - but you did specifically ask about FS. However you don't give a lot of information about your resillience requirements: how reliable is your power? Do you need snapshot capability?

While I'm a big fan of Reiser and ran it for years on my webservers, I wouldn't be using it on a new system - it's never been adopted as part of the Linux kernel, and development is petering out.

So on a Linux box the remaining candidates are XFS, JFS, Ext4 and BTRFS (there's NILFS too - but thats something of a niche tool again, and IMHO again poor on the performance side).

If you must use LVM (even without snapshots IME there's a significant performance overhead) then make sure you've got a recent release with barrier support - and enable barriers if you're using a journalling filesystem.

If performance is your only consideration then it's time for you to start benchmarking - I recommend using Jens Axboe's fio for measuring read/write performance - it is configurable to simulate a number of workloads.

symcbean
  • 21,009
  • 1
  • 31
  • 52
  • The only thing I need is not to worry about inodes and let them being created dynamically. Nothing else. Performances should be broadly the same for reads and seeks. Need to benchmark that statement – Larry Dec 04 '13 at 15:40