4

Assuming the following parameters for a new server storage configuration:

  • All drives are SSD
  • Few large drives are preferred over many small drives
  • RAID 1/mirroring is preferred over RAID 5/6/parity
  • A dedicated hardware RAID adapter is preferred over software RAID or storage spaces
  • Edit: Only one single RAID array is required

So to me it all boils down to a three drive setup with either

  • 2-way RAID 1, with hot spare
  • 3-way RAID 1, no hot spare

While the obvious advantages of the 3-way setup are

  • Potential read performance bonus
  • Concurrent loss of two drives supported

Are there any points to make in favor of the 2-way hot spare setup except maybe these?

  • All data is also written to the third drive, wearing it out like just like the other two, while a hot spare won't be written to before a potential array rebuild.
  • Potential write performance penalty due to all data needing to be written three times opposed to twice (which could be negated by the hardware RAID controller)

Edit: Added parameter to clarify that only one single RAID array is desired

Hannes
  • 51
  • 1
  • 4
  • 1
    I'm curious; which hardware RAID controller did you have a look at, which supports 3-way RAID1? I don't know a single one. – s1lv3r Oct 24 '16 at 14:51
  • 1
    Also, would the spare be available for use by another RAID array, or would it be dedicated only for the RAID-1 mirror? I don't see the use of a spare that's dedicated only for the mirror - if one drive fails, the controller would have to copy everything from the remaining single drive to bring the spare online. During the copy, if the one original mirror drive failed the array would be dead. – Andrew Henle Oct 24 '16 at 15:10
  • @s1lv3r: I have not evaluated if there is one actually. I was just assuming that there should be one if my points are valid. Just as Andrew points out, I cannot see any benefits in using a hot spare for a single RAID array (which is all we need). But if the obvious advantages of a 3-way mirror haven't encouraged the manufacturers to support it yet, then of course that settles the matter, leaving me with the only remaining option of a 2-way mirror with a hot spare. – Hannes Oct 25 '16 at 06:33
  • @s1lv3r HPE's newer P-series controller allow 3-way R1 - essentially you just have three full copies of all the data - not sure how it performs but the main benefit is the lack of impact a drive rebuild has on performance - plus theoretically it can be more resilient. – Chopper3 Oct 25 '16 at 07:12
  • @Chopper3: Thanks for your input. Good to know that there actually are controllers supporting 3-way RAID 1. Do you know if they can be plugged into regular tower servers (not necessarily HP ones)? And if anybody knows of other adapters, please feel free to list them here. – Hannes Oct 26 '16 at 09:17
  • @Hannes - I only know the HPE ones and only know that they work in their servers - no experience with doing what you want to do. – Chopper3 Oct 26 '16 at 19:53
  • @Chopper3 - You're correct, the HPE P-series definitely supports it. They call it "RAID 1 ADM" (Advanced Data Mirroring). Since we are evaluating a HPE server anyway, I guess that might turn the tide in favor of one. – Hannes Oct 27 '16 at 13:19

1 Answers1

3

Edit: Added parameter to clarify that only one single RAID array is desired

In that case you get no benefits. The purpose of a hot spare nowadays is to serve many different arrays/pools. If you have 12 pools out of 24 disks, you just need one or two spares instead of 12 for first response until someone can be at the server and replace the disks.

All data is also written to the third drive, wearing it out like just like the other two, while a hot spare won't be written to before a potential array rebuild.

In my eyes this is a downside, because you only know if sectors are bad on your drive (applies more to hard disks than SSDs) when you start writing to it, which will be the most dangerous and stressful time (for the remaining drive as well as the general load as well as your sanity. Discovering errors just then is something you would like to avoid - predictability is valued higher than less wear and tear. I would make exceptions for things like write-heavy cache devices that wear out the SSDs very fast, or an array of cheap SSDs that should last the maximum amount of time.

Potential write performance penalty due to all data needing to be written three times opposed to twice (which could be negated by the hardware RAID controller)

As long as you have remaining bus and controller bandwidth, it should not interfere. On the other hand, now you can read from three disks instead of two, meaning roughly 50% performance increase (less actually, but this is the general idea). Exceptions apply if the third disk is very slow, but I assume they are similar or identical types.

user121391
  • 2,502
  • 13
  • 31
  • Actually that came to my mind, too. In my eyes it's quite risky to rely on a disabled or at least idle spare device which might turn out malfunctional when a rebuild is required - during which the array is not protected. Initially, I just wanted to mention the potential higher cost of wearing out all devices, but I totally agree that predictability is much higher valued than less wear. – Hannes Oct 25 '16 at 07:59
  • @Hannes For HDDs, it is pretty clear because wearout is a very small theoretical problem, while for SSDs it is much more interesting. But even the SSD TBW values are conservative to reduce warranty claims, so you might add some additional padding after the disk is beyond its predicted lifespan... but then all you pool disks would have the same problem, so I guess it is more a question of perspective: are you interested in general pool health or on the negative effects on the spare itself. – user121391 Oct 25 '16 at 08:54
  • Since the I/O profile is expected to be rather read heavy, I would assume that the drives won't reach their TBW before the end of their warranty lifecycle. Given that enterprise SSD models that come into consideration provide >3,000 TBW as of today, that won't be an issue. It might be an issue for heavy write profiles, I agree. You'd have to swap drives here from time to time, but I'd still prefer resiliency over the hot spare drawbacks. – Hannes Oct 25 '16 at 10:04