0

We have a 24 disk SAN, currently configured in RAID 50, with a RAID 0 stripe across two RAID 5 arrays with 11 disks in each RAID 5. The two remaining disks are allocated as hot spares, one for each RAID 5.

Initial RAID 5 setup with two RAID 5 arrays

I'd like to move this setup to use RAID 50 with three RAID 5 arrays inside the RAID 0 stripe. This increases the amount of disks that can fail before the array is lost, as well as decreases the chance that two disks fail in the same RAID 5 array. It may also have performance benefits.

Desired RAID 50 setup with three RAID 5 arrays

What disadvantages does moving to a greater number of RAID 5 arrays in a RAID 50 setup have? Obviously, you are sacrificing usable capacity for increased resiliency, but are there any other disadvantages going from a RAID 50 with two larger RAID 5 arrays to a RAID 50 with three (or more) smaller RAID 5 arrays?

jduncanator
  • 387
  • 1
  • 3
  • 11
  • 2
    A sane recommendation hinges on what this SAN does. What kind of environment(s) and workloads is this thing supporting? – Spooler Mar 06 '18 at 08:01
  • @SmallLoanOf1M I appreciate the comment, I'm not looking for any recommendations specific to my use case, more so what general disadvantages there might be in using more RAID 5 arrays than less. If this can only be answered properly with more information, I'll happily update my question with specifics of my use case. – jduncanator Mar 06 '18 at 09:25
  • 1
    *checks watch*...yep, still 2018 - you're still using R5/50? Man I hope they're are small disks. – Chopper3 Mar 06 '18 at 10:09
  • @Chopper3 They're 900GB SAS 15K drives, so array rebuild times are not of concern. RAID50 is the recommended array layout that our vendor has recommended to us. From my understanding, RAID50 is still the most recommended RAID layout for large file servers (other than RAID10 which will be cost prohibitive in our situation). If you can come up with an alternative that isn't RAID10, I'm happy to hear it! – jduncanator Mar 06 '18 at 13:03
  • 1
    R5 is dead, has been for nearly a decade, nobody uses it - please don't – Chopper3 Mar 06 '18 at 13:27
  • 1
    *RAID50 is the recommended array layout that our vendor has recommended to us* Your vendor is recommending **eleven** disk RAID5 arrays?!? Unless you're doing large-block streaming writes *and* the RAID controller is really, really, really good and not very loaded, your write performance is going to be horrible unless you've gone through the trouble of matching your write operations to the RAID block size. – Andrew Henle Mar 06 '18 at 17:10
  • @AndrewHenle I understand that RAID 50/5 should be avoided if possible. I was deliberately trying to leave my specific setup out of the question to make the answers more applicable to a greater number of scenarios. We may not use RAID 50 in the end, however I was still after an answer to my specific question, rather than a recommendation on which RAID should be used for my setup. – jduncanator Mar 06 '18 at 22:38
  • Thanks everyone for the comments. I understand that RAID 50 is no where near ideal for my setup, and I am always open for suggestions to ways to improve. However I feel this has gotten a little side tracked. I was looking a very specific answer to my question, rather than general recommendations as to what to use instead of RAID 50. We may not even use RAID 50 in the end (but it was the next best thing our controllers supported after RAID 10 which is prohibitive in our install). If you know of any specific disadvantages of using more RAID 5s in a RAID 50, please leave an answer. – jduncanator Mar 06 '18 at 22:45

1 Answers1

1

Moving this raid-50 array to use smaller subsets of R5 would present a few advantages. R5 doesn't scale well past five or so drives in most implementations, with sharply decreasing returns and sometimes worse performance with (let's say nine) drives rather than five. Eleven is pushing it, but these numbers greatly depend on the controller you're using.

With each R5 moving more efficiently and predictably, the R0 can combine their performance easily and you should see measurable performance differences between the implementations. And as you mentioned, you'll be able to suffer more drive failures.

However, I would generally advise to avoid RAID-5 arrays. R5 has several serious flaws that affect whole-array integrity (see about the RAID-5 write hole), the most important of which is due to the design concept itself - an array with a single missing member is degraded, and it only takes a single error in that degraded array to damage data or bork the entire array. A single data error either across an entire drive or across a more limited area is quite likely statistically when operating many large disks in an array. Obviously hot spares mitigate this issue, but you're still leaving a rather large hole for bad things to happen during rebuilds or power failures. Additionally, R5 performs quite poorly when degraded and is pretty sub-optimal for random write heavy workloads even when not degraded.

Raid-10 would be a lot more stable and would perform much faster, but has obvious expense concerns. Raid-60 could be a great middle ground depending on your workload, and wouldn't seriously impact your storage costs. Using two R6s would allow for four drive failures without one failure being a potential imminent catastrophe, and R6 will perform much better when scaled to that amount of drives.

Spooler
  • 7,046
  • 18
  • 29
  • Thanks for the answer, I'm aware of the advantages, and I'm also aware of all the pitfalls and risks of RAID5 (RAID10 would be my first choice, but is not applicable in our scenario). I was specifically interested in if there were any disadvantages in moving to a greater number of RAID 5s over the existing two (with 11 spindles in each). – jduncanator Mar 06 '18 at 13:11
  • I can't think of any disadvantages to reducing the size of your R5 arrays or using more of them within your R0. If that's really for some reason your only option, I suppose go for it. – Spooler Mar 06 '18 at 18:17
  • So the answer is then "there are no disadvantages, only advantages". If that's that case, could you update the answer to include that as the first line and I'll accept your answer. Thanks! – jduncanator Mar 06 '18 at 22:45
  • The answer is what I wrote. I can't speak to any explicit disadvantages because I don't have enough information on your environment, the raid controller, or the nature of your workload pattern to identify disadvantages. The primary content is still "don't use raid 5", as raid 5 itself is the primary disadvantage. Your vendor is out of their minds to recommend using it so explicitly. – Spooler Mar 07 '18 at 01:08