4

In the past, servers I've put together have all had 8 or fewer disks, so I've not had to give the storage controller much thought - a standard controller like the P440ar (my experience is with HP) that supports up to 8 disks is sufficient.

The server I'm currently spec'ing up will likely have between 10 and 12 SFF disks, meaning the usual standard P440ar will not be sufficient. To support >8 disks, it seems I have a couple of options:

  1. Stick with the P440 and add a SAS expander card
  2. Switch the controller to something like a P840 that supports up to 16 disks without a SAS expander card

Therefore, I've been trying to understands the pros/cons of each approach.

I appreciate that you have no option but to use a SAS expander if you want to create a RAID array that spans more disks than the controller itself supports. Assuming that's not the case, what are the pros/cons/differences between:

  1. Smaller controller + expander
  2. Larger controller

(where 'smaller' is a controller that doesn't support the required number of drives without an expander, and 'larger' is a controller that does)

... and when/why would one choose one option over the other?

dbr
  • 1,852
  • 3
  • 23
  • 38

2 Answers2

3

Based on my own experience with expanders I would go for better controller for few reasons:

1) Price of expander + controller comes close to one more expensive controller.

2) You have to deal with compatibility issues between controller and expander and then also cable issues (right cable between controller and expander) then disk cables between expander and disks. They aren't all born the same.

3) Flashing expander is painful, you need windows port or original linux utilities e.g. sg_write_buffer and lsscsi/sg_scan.exe and that's already good reason to avoid it (unless you buy both of them from same manufacturer and I know that HP sometimes requires HP mobo).

4) In 12G arena has quite some incompatibilities between expander and controller of different manufacturers (e.g. LSI and HPE). I've heard that identical configurations work for some people but not for others.

Maybe consider two RAID controllers if it suits you, you might get same price and better performance.

PS: if you decide to go with spearate manufacturers get familiarized with flashing expander before you buy it. I suggest windows utilities that come with Intel expanders to start with (sg_write_buffer.exe, sg_scan.exe, update.bat (need to modify it for non intel ones) and they also give RPMs for linux or this article to see what you are getting involved into.

Peter
  • 146
  • 5
0

Theoretically, a SAS expander may be limiting throughput. If you're using a SAS RAID controller with SFF-8087 4-lanes connectors (quite typical), the SAS expander will multiplex a number of disks on these 4 lanes, while without expander each disk will be driven by only one lane.

However be aware that many RAID controller with more than 8 SAS lanes actually use an on-board SAS expander (I don't know the technical details of P840); in which case using an internal or external SAS expander will make little difference, if any.

Raw numbers follow:

  • SAS-2 (6 Gbps), expander, 16 drives: throughput 3 GB/s
  • SAS-2 (6 Gbps), native, 16 drives: throughput 12 GB/s
  • SAS-3 (12 Gbps), expander, 16 drives: throughput 6 GB/s
  • SAS-3 (12 Gbps), native, 16 drives: throughput 24 GB/s

As you can see, the SAS expander may severly limit maximum disk throughput. However, keep in mind that most RAID controllers and disks combinations absolutely can't reach even the low values. If you plan on using 7200 RPM drives, a SAS expander is not the limiting factor. On the other hand, with SSDs it may easily become a point of contention...

wazoox
  • 6,918
  • 4
  • 31
  • 63