2

I've experimented with different striped volumes in vinum under FreeBSD 8.2-release. I have current hardware (Intel i5 processor, adequate MB and 8GB RAM).

My test-setup consists of 2 or 4 disks in a volume, each disk is 2TB, "Green" line Seagate and Western Digital; i.e. consumer disks at 5400RPM. These are in my server, while my desktop that's connected to it (via Gbit LAN) has a quick SSD.

My problem is that I get better performance (i.e. read/write speeds) over my Gbit LAN when I access my single "Black" Western Digital 2TB, than when I access my vinum volumes.

  • Black WD single disk: read = write = 90-105 MB/s
  • vinum volumes, all setups: read = 80 MB/s ; write = 50 MB/s

The vinum configurations I've tried are:

test1:

drive a device /dev/ad32
drive b device /dev/ad34
drive c device /dev/ad26
drive d device /dev/ad28
volume test1
  plex org striped 512k
    sd length 2048m drive a
    sd length 2048m drive b
    sd length 2048m drive c
    sd length 2048m drive d

test2:

drive a device /dev/ad32
drive b device /dev/ad34
volume test2
  plex org striped 512k
    sd length 1907728m drive a
    sd length 1907728m drive b

I've tested test1 with different stripe lengths (512k, 64k, 2048k), but the speeds are the same. Also, I notice no difference in speeds between test1 and test2, which I find very strange, considering these are striped volumes and disk-count therefore should matter.

Does anyone have a clue why this happens and what I can do to tune/speed-up vinum to at least outspeed my single HDD? Should I consider other options for performance (ZFS w/RAIDZ etc)?

More info (edit)

dmesg about the disks:

  • ad26: 1907729MB [ST2000DL003 9VT166 CC32] at ata13-master UDMA100 SATA 6Gb/s
  • ad28: 1907729MB [ST2000DL003 9VT166 CC32] at ata14-master UDMA100 SATA 3Gb/s
  • ad32: 1907729MB [WDC WD20EARX-00PASB0 51.0AB51] at ata16-master UDMA100 SATA 3Gb/s
  • ad34: 1907729MB [WDC WD20EARX-00PASB0 51.0AB51] at ata17-master UDMA100 SATA 3Gb/s

All disks are connected to onboard SATA-controller on my motherboard; Asus P8P67 Deluxe.

When testing ad32 as a single disk, I get write-speed at about 60-70MB/s, read-speed at about 80-90MB/s.

All testing done with one large file of about 2.5GB.

The point: my disks perform slower in a striped volume than alone... Suggestions?

poplitea
  • 457
  • 1
  • 5
  • 16
  • 1
    Your problem could be those "green" model disks. What performance do you get out of them? – MikeyB Oct 07 '11 at 01:25
  • @MikeyB In fact, I haven't tested them as single disks yet, but if the fact is that they're slow, couldn't I at least expect faster speeds with an array of 4 green disks than an array of 2 green disks; both arrays striped? – poplitea Oct 07 '11 at 01:29
  • 2
    Let's get all the information out - individual disk speed, which disk is which, type of controller (SAS? SATA? Onboard? Expansion card?) – MikeyB Oct 07 '11 at 01:33
  • @MikeyB I suppose that's better :-) Now I've edited my post with more info. – poplitea Oct 07 '11 at 01:46

1 Answers1

3

Based on what you've posted so far my guess is that your striped volume is almost certainly being hampered by slower drives (I see one 6Gb/sec link and 3 3Gb/sec links - I'm willing to bet the 3Gb/sec drives also have lower general operating specs.

Like @MikeyB said run speed tests for each disk individually - this will confirm my theory (or send you looking in another direction).
You can expect to see performance bounded by some function of the slowest disk in the array, with a hard upper limit at-or-around the SATA controller's maximum throughput.

You can also try running gstat when testing the array and see what kind of activity you've got on each disk - If the slower ones are maxing out that could be your culprit, and if the controller is pinned that's definitely an issue.

voretaq7
  • 79,879
  • 17
  • 130
  • 214
  • Thanks a lot! `gstat` is a useful tool I didn't know about. I found that ad26+ad28 performs fast in a 2-disk striped volume. About 120MB/s write and 80-90 MB/s read (strange with write>read?). In that setup ad26 is 20% busy, ad28 30%; according to `gstat`. I suppose this is because of the controller; 3 vs 6 Gb/s. I'll experiment further with single disks and test the controllers... – poplitea Oct 07 '11 at 02:23