9

I'm formatting a drive using NTFS which will be dedicated as a file share for users to centrally store their files. The files will likely be large (10's to 100's of megabytes).

Someone suggested that using a large allocation unit size than the default 4k (e.g. 64k) will make it perform better. I think I understand the basic principle behind it, but I'm not sure if its valid in practice. Will this truly make a difference or is this something that may cause more problems than it solves?

dso
  • 553
  • 2
  • 6
  • 11

3 Answers3

9

A larger allocation size will increase performance when using larger files. If they are all going to be large files then it might pay to increase the allocation size to 32KB or 64KB.

Be aware that the larger the allocation unit size, the more disk space that will be wasted. This is true regardless of the sizes of the files stored on the volume. If the allocation unit size is 64K and you save a 50K file, 14K will be wasted. If you save a 800K file, it will be divided into 13 chunks, but the 13th chunk will only have 32K of data resulting in 32K of wasted disk space.

A resource for performance tuning of NTFS drives can be found here: https://web.archive.org/web/20090724155321/http://www.windowsdevcenter.com/pub/a/windows/2005/02/08/NTFS_Hacks.html

Good luck, any further questions dont hesitate in asking.

Lima

Slix
  • 105
  • 2
Lima
  • 374
  • 3
  • 12
  • Just remember that it will cause files to use more disk space (it will round file sizes up to the nearest full block), so if disk space is an issue or if there will also be smaller files, you should consider carefully. – Catherine MacInnes Jul 02 '09 at 15:24
  • 1
    It's absolutely correct to observe that "files [will] use more disk space" however it's not accurate to focus on whether there will be smaller files, but rather _how many_ files will end up on the volume. Since each file is rounded up to the nearest block size, regardless of that file's total size, it's the sheer number of files that can result in large amounts of wasted space. – I say Reinstate Monica Sep 11 '14 at 21:18
  • 1
    To add to @Twisty's comment: expected wasted space = expected # of files * allocation unit size / 2. So if I choose an allocation unit size of 64k (65,536 bytes), and my files are all huge so I only have about 3,000, I can expect to waste 3,000 * 65,536 / 2 = 98,304,000 bytes, or about 98 mb. This waste is known as [internal fragmentation](https://en.wikipedia.org/wiki/Fragmentation_(computing)#Internal_fragmentation) (thanks, required CS classes). – Jake Mar 10 '16 at 15:14
5

Setting the allocation block size can improve performance for accessing large files, but it is unlikely to improve the performance of a network fileshare noticeably as other bottlenecks will dwarf any local benefit.

There are some things to look out for:

  • files will take more space, so if you have a lot of small files this will be a problem
  • accessing small files may be slower as the system reads whole blocks at a time (so reading 64Kb for a 1Kb file if you use 64Kb blocks), though depending on the read-ahead behaviour of your drives this may not be noticeable
  • you might find that it actually harms performance when the access pattern is very random and/or there are many concurrent processes accessing the resource over the network

My gut suggests that you would not notice much benefit (or detriment) performance wise in must use cases, and my gut is quite large so I don't tend to argue with it, so I would stick with smaller cluster sizes for space use efficiency.

David Spillett
  • 22,754
  • 45
  • 67
0

I think the general idea is that larger = better performance at the cost of disk space.

There is a rumble online that changing the default size will cause badly coded disk utilities to error or fail, so you might want to keep that in mind if you plan not to plan for backups ;-)

MathewC
  • 6,957
  • 9
  • 39
  • 53
  • 1
    Stay clear of non-standard disk utilities, except in disaster recovery operations that cannot be performed with standard tools. Planning not to plan for backups is planning for disaster. – thecarpy Sep 24 '15 at 08:55