8

I have been working on my NAS that is using server 2012r2 with storage spaces.

  • the drives are off the shelf 7200 rpm drives
  • the drives are bitlocker encrypted
  • the drives are in a 8 disk parity layout

My question is this. Using SMB multichannel I have been able to transfer a 500 MB file in about 2.5 seconds at about 200MB a second to my NAS, but I see that the NAS copies that file to disk over the next 10 seconds. Why does it take that long to copy to disk? Surely 8 disks in a storage space could write more then 20-40MB a second...

Network Utilization on top, Disk Utilization on bottom

The problem comes when I try to write huge files 3+ GB to the server as the transfer speeds will go from 200MB/s down to almost 0 while it writes to disk everything that has been cached.

peterh
  • 4,953
  • 13
  • 30
  • 44
Hunter Brelsford
  • 119
  • 1
  • 2
  • 2
    Have you tried to disable Bitlocker and re-test your storage? – Strepsils Jul 31 '17 at 19:49
  • 1
    That is expected behavior, actually. As Microsoft states in their official documentation, parity layout can be used only for archiving tier which assumes the user is fine with that kinds of speeds. Please check exactly the similar case here with the newest Windows 10/Windows Server 2019 Storage Spaces version https://datacenteroverlords.com/2018/12/17/microsoft-storage-spaces-is-hot-garbage-for-parity-storage/ – Net Runner Apr 13 '20 at 08:12

2 Answers2

8

This postponed write happens because of a a) file system level write-back RAM cache, and b) parity Spaces design, which absorbs all the writes to the internal log initially to avoid “read-modify-write” on the “critical path” dramatic slowdown. When your write is complete file system keeps flushing data in the “lazy writer” worker thread and Spaces keep reading the internal log and doing actual writes in the background as well.

You can speed the process up by a) providing dedicated flash devices for Spaces log, and b) avoiding parity and going with mirror-stripe (journaling flash will help here as well). See:

https://mrickert.com/2016/04/25/adding-journal-drives-to-windows-storage-spaces-2012/

Another option is to upgrade to Windows Server at least 2016 (2019 is recommended due to tons of Spaces related bugs fixed there) and use newly represented (Rebranded? Code is there for years...) Mirror-accelerated-Parity design. See:

http://knowledgebase.45drives.com/kb/kb450193-creating-mirror-accelerated-parity-volumes-and-storage-tiers-in-storage-spaces-windows-server-2019/

BTW, it’s bad idea to use file copy to measure your storage performance, see here why:

https://blogs.technet.microsoft.com/josebda/2014/10/13/diskspd-powershell-and-storage-performance-measuring-iops-throughput-and-latency-for-both-local-disks-and-smb-file-shares/

Good luck! :)

BaronSamedi1958
  • 13,676
  • 1
  • 21
  • 53
  • 3
    Nor WS2016 neither WS2019 will change this behavior https://serverfault.com/questions/757693/storage-space-write-cache-and-write-speed/1011850#comment1314608_757693. It is expected. And as Microsoft documentation confirms, parity layout should be used only for archiving where those speeds are OK (what?!). Mirror-accelerated parity solves the issue somewhat but has its major problems too, which are Storage Spaces Direct and ReFS are required. And both of these technologies aren't reliable at the moment to say at least. – Net Runner Apr 13 '20 at 08:21
0

storage spaces ensures to have a consistent state. parity in general is the slowest of all options - you don't get any performance improvements.

if you're willing to tolerate dataloss in case of an outage (or other weird crashes) you may enable powerprotected mode on the space

Set-StoragePool -NameOfPool -IsPowerProtected $true

edit: you may find more about the underlying concept when you dig into how storage spaces handles metadata flushes (which is done in sync unless you're on powerprotected mode)

when i remember correctly you may use SSDs (at least two) to be used as journaling disks for the parity layout. a good introduction is available here: http://social.technet.microsoft.com/wiki/contents/articles/15200.storage-spaces-designing-for-performance.aspx

Daniel Nachtrub
  • 1,022
  • 7
  • 12
  • 2
    Power protected mode is already enabled. I understand there isn't any additional caching or anything in parity mode but still the drives should be able to do a sustain write of at least 20-50MB a piece indefinitely right? So 8 drives times 20-50 is what i would expect my sustain write throughput to be... – Hunter Brelsford Feb 19 '16 at 23:26
  • I think what im going to do is run some tests on the local volume created by my storage space and see how it performs not just through an SMB file transfer. Ill post results. – Hunter Brelsford Feb 19 '16 at 23:34
  • 1
    how's read performance via smb - is this also affected? in my experience parity mode on 7.2k drives deliver around 35-40MBs throughput on large data sets without powerprotected mode. in some installations we see around 80-90MB/s while powerprotected is enabled (without even using smb multichannel). – Daniel Nachtrub Feb 20 '16 at 12:53
  • Well i cant comment on the pre multi channel configuration because my NAS has teamed nics, But the client that was writing to it was writing at 100MB/s. What i see is that if the file is over about 2-2.5 GB's it writes at MAX network speed until that point then it slows to a halt and has to flush to disk. – Hunter Brelsford Feb 20 '16 at 20:10