1

My office has been using a FreeNAS server that was built with desktop hardware. We use it for storing RAW photos (~50mb each). We have 3 workstations that access (browse, retouch, save) the files though the local network. We are interested in upgrading the system with a new motherboard for a few reasons, but there are also a couple of issues we foresee in switching to a newer board. Currently the server has an ASUS P5B Deluxe motherboard with 4GB of DDR2-800 memory.

Problems with the existing motherboard:

  1. they're getting harder to find. We already had to replace the board once about a year and 1/2 ago when the original board blew some capacitors.
  2. memory speed / capacity is limited at DDR2-800 / 8GB (4x2GB) and we understand XFS will require more.
  3. no onboard video, meaning we have to have a PCIe video card which is not needed for a system that sits in a closet with only power & network cables attached.
  4. SATA2 controllers (no SATA3...)

Our server chassis has (12) SATA 3.5" bays (not port multiplier backs). The first 6 bays are connected to the motherboards onboard SATA2 ports, and the rest are connected to (2) PCI SATA2 controller cards with (4) ports each. This gives us a total of (15) SATA2 ports - obviously more than the server needs.

We would like to upgrade the system to SATA3, with more memory, and run XFS file system. We would also like to use a smaller form factor board (e.g. Mini ITX) if possible, that has onboard video, can take up to 16GB of DDR3 memory, and can support at least (12) SATA3 ports without use of port multiplier boards.

Here's the dilemma: I have been looking up motherboards that meet the above, however they seem to have only (4) to (6) SATA6 ports at most. The smaller boards also tend to lack expansion ports - usually only having (1) PCIe 16x slot.

So, two questions:

  1. if we move to a Mini ITX board that has (4) onboard SATA3 ports, what would be the best way to get the other (8) bays connected to the board.
  2. considering that we are using 7200rpm "Green" SATA3 drives, is it even necessary for each drive to have an independent SATA3 channel or would port multipliers still allow enough bandwidth for the drives without bottle-necking.

If the answer to question #2 is "You don't need SATA3"...I was thinking of using something along these lines:

Thanks for your time and thoughts.

Tom Frascone
  • 41
  • 3
  • looks like a shopping list question ..... – mdpc May 30 '13 at 19:43
  • @mdpc yes, I thought that too, but the two questions he has aren't shopping questions. The answer to the first would be to buy a proper server rather than bodge an array and the second...green drives? Are you using them in RAID? Are you mad? – tombull89 May 30 '13 at 19:45
  • 3
    Green drives? And you haven't lost all your data yet? You are very lucky. Go buy a lottery ticket before your luck runs out. – Michael Hampton May 30 '13 at 19:45
  • 1
    Stop using sata port multipliers. In fact, stop using using sata ports. Connect them all to a SAS controller. http://www.newegg.com/Product/Product.aspx?Item=N82E16816151048 – Zoredache May 30 '13 at 19:53
  • @MichaelHampton Yeah, this is NOT a shopping question. I can shop on my own... We are a small business on a budget - hence the reason we used desktop hardware to build a NAS in the first place. Also, we were using a mix of Western Digital "Blue" drives in the server and we had two fail within 6 months. Ever since switching to the "Green" drives we haven't had a single failure in about 2 years. The drives are configured so the data is redundant so the two drive failures we had in the past did not cause any data loss. – Tom Frascone May 30 '13 at 19:54
  • @Zoredache - That controller card costs more than our entire server as it now sits... As I mentioned before, we are a small business and we are looking to upgrade within our modest budget. – Tom Frascone May 30 '13 at 19:55
  • @MichaelHampton I have to ask - what's the problem with Greens? I know they're not exactly optimal for a storage cluster, but actually dangerous? – Mikey T.K. May 30 '13 at 19:59
  • Since you can shop on your on your own, then search around for an alternative SAS controller. SAS controllers are the easiest way to get lots of ports. – Zoredache May 30 '13 at 20:00
  • 1
    @MikeyT.K. Greens have a rapid "spin down" time, so if you've got greens in a RAID volume and one spins down the RAID craps itself and marks the drive as failed, leading to all sorts of fun and games. – tombull89 May 30 '13 at 20:07
  • @tombull89 - We have been running the Greens in Software RAID1. Have never had any issues as you describe. Perhaps that issue is with Hardware RAID configurations...? – Tom Frascone May 30 '13 at 20:11
  • Food for thought - http://www.synology.com/index.php?lang=default + NAS drives (WD Red) + iSCSI target (optional) will give you great flexibility without cost or upgrade hassles for the server. You could even run the NAS as a file share, but for me iSCSI target is great for multiple servers on the same drive set – morleyc May 30 '13 at 21:16

3 Answers3

6

You could reword this as a non-shopping question, "How do I hook up a dozen SATA drives without using port multipliers or multiple PCI cards?" so I'll write an answer to that magical question:

Buy a multi-ported SAS card and associated SAS 4x breakout cable (SFF-8087 MiniSAS to 4 SATA). 6gbps SAS2 is like SATA3 (6Gbps). Then use whatever motherboard you want. If you don't need hardware raid you could consider something like the LSI 9201-16i (16 SATA devices) but there are all sorts of options.

notpeter
  • 3,515
  • 2
  • 26
  • 44
  • Thank you @notpeter - I thought something along these lines might be a good option, but the next issue is doing this according to our budget. The card you suggested (with cables) is at the top end, but I will shop on my own to see what other options are out there. – Tom Frascone May 30 '13 at 19:58
  • Well, if you truly only need 8 additional ports (on top of the 4 from the motherboard) you can get by with two SAS 4x channels instead of four. IBM M1015 is an LSI rebadge and can be had for $100. – notpeter May 30 '13 at 21:09
0

You can find on ebay dozens of cheap RAID cards. Do not buy a single big RAID card to not have a Single Point Of Failure, but buy rather 2-3 cards with 4 SATA ports each. SAS would be nice but it's not a must have.

WD Greens are not really suited for your use. Buy rather blacks or reds, or the enterprise ones.

Do not buy a Mini ITX board but rather a micro ATX board, it's still much smaller than ATX boards but the microATX boards have 3-4 PCIe slots instead of 1-2 on miniITX. Buy 2x 8GB and you should be fine for your usage. 32GB would be a must for the storage space you intend to use only if you want to have the supaa nice / bleeding edge / high end features of ZFS.

Avoid by all means ports multipliers. I speak from personal experience. It's not well supported, and generally induce more bugs than it solves.

Yannovitch
  • 309
  • 1
  • 8
  • Having multiple raid cards usually does NOT solve the single point of failure issue. Imagine a 12 disk RAID6 array with 3 RAID cards with 4 disks on each, and one of the cards dies, you have simultaneously lost 4 drives on your array that can only handle the loss of 2 drives, the whole array is down. The only way to arrange it so the array does not crash with a single card failure is to have it so losing a card does not lose more than the max hdd loss in a redundancy group of a raid 10, 50, 60, or similar array. So for RAID10 have max 2 cards, and max 1 hdd in a mirror per card. – BeowulfNode42 Oct 26 '16 at 07:41
0

I love freeNas but..

If you are open to alternatives, I would recommend moving away from commodity desktop hardware, and twards a more specialized solution. You will find the setup and maintenance is much easier, and the failure rate is lower.

Having researched every SOHO on the market, and having worked with raid, I think a small dedicated NAS would be your best bet. Here is the one I recommend for home and small offices because the software is super simple to use, you don't need to be a sys admin to set it up.

http://www.synology.com/products/product.php?product_name=DS1813%2B&lang=us

If you wish to stay with building your own server, You will get much better performance by staying away from the green drives. Also stay away from port multipliers.

spuder
  • 1,725
  • 3
  • 26
  • 42