6

I have a single Windows Server 2016 Standard Server, with 3x 4 TB drives, on plain SATA Controllers (2x On-Board Intel, 1x Add-On PCI Express Asmedia 1061). The server itself is virtualized, running on a Windows Server 2016 Standard in Hyper-V, with the disks being physically attached to the VM in Hyper-V.

I could create a regular RAID-5 Volume in Computer Management, which has been supported for several Windows Server versions (Back to Server 2000 or at least Server 2003?)

New RAID-5 Volume

However, the big new Storage Feature since Server 2012 has been Storage Spaces, which offers Parity.

However, I haven't been able to find much information about why I would use Storage Spaces Parity over a regular RAID-5 Volume in a 3-disk/single parity setup. I've seen some discussion about more disks and using multiple redundancy (instead of just protecting against a single disk failure), and I've seen tons of discussion about Storage Spaces Direct (which seems to be the only thing anyone on Server 2016 seems to talk about).

Has anyone done any actual experience with Storage Spaces over regular RAID-5 Volumes and could advice which one would be a better option?

(Better, as in: If a drive dies, I just want to replace it and rebuild automatically, and if I have to move two working drives to another Windows 2016 server for data recovery, it just works.)

Michael Stum
  • 4,050
  • 4
  • 36
  • 50
  • (The use case is a simple file server for a handful of users. I prefer my storage to be as boring as possible (which is why I'll go with NTFS over ReFS). And yes, backups do exist, at least once per day.) – Michael Stum Jul 22 '17 at 18:20
  • 2
    Most importantly PLEASE don't create a R5 array with large disks (anything over 1TB) - you WILL lose data - it's something we bang on about to everyone on here but there's no place for R5 on big spinney disks in 2017. – Chopper3 Jul 22 '17 at 18:26
  • @Chopper3 Can you elaborate? I understand that with bigger disks, the risk of having up-to-date backups decreases, but in the balance between cost and needing the extra space, RAID-5 seems like it adds the reduncancy of "One drive can just die". In understand that RAID is an availability mechanism since no RAID is going to protect against Ransomware or data deletion. So just wondering: Is there something inherently wrong with R5, or is it just the reality of "More than 1 disk WILL fail, eventually"? – Michael Stum Jul 22 '17 at 18:29
  • 2
    To answer myself: The problem with RAID5 is that rebuilds not only take forever, but also stress out the remaining drives so much that it's likely for one of them to fail (since for every block that needs to be rebuilt, all blocks in that stripe needs to be read - across all drives. So rebuilding a failed 4TB drive requires reading the full 4TB from all other drives, each). With 3 Disks, Raid 1E would be much better, and a 4 disk RAID10 would be much preferred. – Michael Stum Jul 22 '17 at 18:47
  • Going to pick up an [LSI SAS 9211-8I](https://www.broadcom.com/support/download-search/?pn=SAS+9211-8i+Host+Bus+Adapter) controller. Can't put in a fourth drive (space constraints in the server) and can't rebuild the entire server right now (though that's planned next year for other reasons). – Michael Stum Jul 22 '17 at 19:11
  • 1
    also read this, there's lots of similar - more detailed and more up to date - articles if you look around but this is quite easy to follow; http://www.zdnet.com/article/why-raid-5-stops-working-in-2009/ – Chopper3 Jul 23 '17 at 13:49

3 Answers3

7

It's a bad idea to use parity Storage Spaces because:

1) Single parity is dangerous: every time one disk dies and you start a rebuild process there's a heavy load applied to all remaining spindles, so there are great chances you'll get second, now deadly fault.

2) Performance is horrible. I mean it! ZFS has proper journaling and variable sized parity strips, while Storage Spaces have none.

Use RAID10 equivalent or a single node Storage Spaces Direct + ReFS and a multi-resilient disks.

https://social.technet.microsoft.com/wiki/contents/articles/11382.storage-spaces-frequently-asked-questions-faq.aspx#Controlling_the_Number_of_Columns

(that's for performance to build a proper RAID10 equivalent)

https://charbelnemnom.com/2017/06/how-to-create-a-multi-resilient-volume-with-refs-on-standalone-server-in-windows-server-2016-ws2016-hyperv-storagespaces/

(that's for multi-resilient disk, one will give you flash-in-mirror + disks-in-parity)

BaronSamedi1958
  • 13,676
  • 1
  • 21
  • 53
  • 2
    Thanks. Can't do Storage Spaces Direct (as it's not included in Standard) or add more than 3 drives (no space in case, planning to rebuild though) so my next approach is RAID 1E with a RAID Controller. Thanks for the pointers though, it seems that Storage Spaces Direct is really the preferred software-defined storage solution for Windows now. – Michael Stum Jul 24 '17 at 05:55
  • 4
    Get StarWind vSAN free (1) or HPE StoreVirtual (2) and do mirror on top of a "hardware" dual parity RAID6. The same approach is utilized by HPE|SimpliVity to have better performance and fault tolerance on just a pair of "controller" nodes. 1) https://www.starwindsoftware.com/starwind-virtual-san-free 2) https://www.hpe.com/emea_europe/en/storage/storevirtual.html – BaronSamedi1958 Jul 24 '17 at 13:54
7

Unless you're doing a HEAVILY read-oriented system, Storage Spaces Parity mode is less than optimal. I'd strongly suggest using the Mirror mode. Do note that Mirror in Storage Spaces is NOT RAID1. It functions like RAID1E (mostly). It will divide your disks into chunks, then ensure all data exists on 2 disks (for 4 disks or fewer), or 3 disks (for 5 disks or greater). When combined with ReFS, with the integrity streams enabled and enforced, it will also checksum your data like ZFS does.

Also, I think you're confusing Storage Spaces with Storage Spaces Direct.

Windows Server 2016 Standard has Storage Spaces, but not Storage Spaces Direct. You do NOT need anything offered with the "Direct" as you're not doing clustered storage. There's a reason it's only offered in the DC version: it's not useful outside of a clustered scenario.

You can absolutely open up Server Manager and create a 3 disk "Mirror" pool, which will function like RAID1E (mostly), and give you 6TB available, rather than 8TB as you would get with Parity mode, but you get much better write performance, and better resiliency. You can add a 4th disk later and rebalance the pool to being more like RAID10 (2 columns, 2 stripes).

The RAID5 stuff in Disk Management is garbage, do not use it.

Daniel Winks
  • 366
  • 1
  • 2
  • It seems your answer is the only one that actually answers his question, "which one should I use?", instat of saying you have to go for raid 10, while he only has 3 drives. Now you say "The RAID5 stuff in Disk Management is garbage, do not use it." can you tell me why this is? – ikwillem Feb 13 '18 at 15:28
  • RAID 5 in the "Dynamic Disks" section in Disk Management in Windows has a number of serious flaws. First is performance. It's bad. Very bad. Secondly is implementation: the primary disk (the OS volume, ie C:\) contains all of the information about the dynamic volume. If the primary disk fails, you also lose the RAID 5 volume. Effectively this makes it RAID05, since you're at risk for losing everything if you lose just a single disk. There's more reasons than just those two to stay away from Dynamic Disks in Windows, but those should be enough. – Daniel Winks Feb 15 '18 at 15:45
  • Ok thanks, last year I bought 4 hdd's for raid-5 in storage spaces, after installation I wanted to test what would happen if one drive failed so i pulled out 1 of the sata cables. I got a error message all well and good, but when i tried to remount the drive, it wasn't picked up automaticly and i could not remove the failed drive from storage spaces. A fresh windows install was the only thing that worked. After a new raid-5 my computer froze while coping data and the only way i could get the drives to work was with a raid-5 in disk management. Also last week i did a fresh install no problems. – ikwillem Feb 17 '18 at 09:29
  • Also my performance seemed to be better with the disk management raid-5, with storage spaces (before pulling out the sata) the write speeds were very inconsistant, dropping to 0 mb/sec, with disk management this was gone. I initially though storage spaces where highly expiremental and in some kind of beta stadium because of this. So I am very curious how you came to exactly the opposite conclusion. – ikwillem Feb 17 '18 at 09:41
4

1) There is nothing inherently wrong with hardware RAID. RAID 5 has gotten a bad rap lately because disk sizes are increasing rapidly which is making for very large arrays and increasing the mathematical likelihood of unrecoverable array failure.

2) "Software RAID" like Storage Spaces comes in a lot of flavors and configurations. Some are bad, and some are quite good. This is ultimately a mixture of hardware and software that needs to be properly configured.

Why use "Storage Spaces" or ZFS vs a RAID controller: If you make a very large RAID array (we'll say something like 4x4tb RAID 5) your likelihood of a puncture (this simply a bad bit on an otherwise functional disk) is quite high. If you're using only a hardware RAID controller, the controller has no idea what you are or are going to install on the disks (nor does it care). It's simply using an algorithm to bond those disks into one big "physical" disk to your operating system. This is handy in a generic sense but in the event of drive failure you could lose an entire disk and then one of your GOOD disks could have a bad bit and cause a rebuild failure, costing your your precious, precious data. The controller has no idea what this data is, so it can't really help you recover partial data either (because it doesn't speak Windows or Linux or whatever) so it just says 'sorry for ya'.

If you run software RAID, you're going to need a LOT more hardware to manage the data movement that would normally be handled by a specialized chip on your RAID controller but in the event of a puncture, ZFS or ReFS (Storage Spaces) can at LEAST give you SOME data back.

RAID 5 vs RAID 6 vs RAID 10 etc: RAID 5 is getting a bad rap now because of what was just described. It is now said that in every 32TB of spinning rust you're essentially PROMISED a puncture. So, if you're running that 4x4tb RAID 5 and a disk goes bad, you really only have about a 50% chance of getting your data back during a rebuild if you're running hardware RAID! Pretty bad. Even a RAID 10 configuration of equivalent size (6x4tb) would likely take out half of the array in the rebuild process.

Now, SSDs don't suffer from punctures! SSD memory has kind of an internal RAID on each drive so you should never encounter a conventional 'puncture'. Multiple disk failure is still an option but FAR less likely (we'll all have to wait till the sizes get up there to really test this though). So you're much safer (and faster) with hardware RAID here generally.

TL;DR: Use Storage Spaces because disks fail a LOT and storage spaces with ReFS has extraneous parity and rebuild options. Same with ZFS. That said, FreeNAS/ZFS is significantly faster than Storage Spaces for various reasons (mostly that Storage Spaces wasn't really designed for a single computer). Use ECC memory! Backup important data!

thelanranger
  • 139
  • 7