2

we are using a 530-8i RAID adapter,which is none-cache supported, on a Lenovo ThinkSystem SR590 server.And we have 4 HDD(600GB/10K RPM/SAS) in the array. I read an article about calculating the disk/disk array IOPS,our disk array should get IOPS like 130 ,theoretically. I did a test with a tool called CrystalDiskMark,but can only get about 20 IOPS when doing SEQ1M write test. Is this a normal performance of a raid adpter without cache?If not,how can I improve the performance?

The output from CrystalDiskMark:

  • MB/s = 1,000,000 bytes/s [SATA/600 = 600,000,000 bytes/s]
  • KB = 1000 bytes, KiB = 1024 bytes

[Read] SEQ 1MiB (Q= 8, T= 1): 521.796 MB/s [ 497.6 IOPS] < 16036.35 us>

SEQ 1MiB (Q= 1, T= 1): 466.978 MB/s [ 445.3 IOPS] < 2242.63 us>

RND 4KiB (Q= 32, T= 1): 9.169 MB/s [ 2238.5 IOPS] < 14248.25 us>

RND 4KiB (Q= 1, T= 1): 1.219 MB/s [ 297.6 IOPS] < 3352.98 us>

[Write]

SEQ 1MiB (Q= 8, T= 1): 23.278 MB/s [ 22.2 IOPS] <346183.31 us>
SEQ 1MiB (Q= 1, T= 1): 20.761 MB/s [ 19.8 IOPS] < 50402.19 us>

RND 4KiB (Q= 32, T= 1): 2.203 MB/s [ 537.8 IOPS] < 58681.18 us>

RND 4KiB (Q= 1, T= 1): 0.751 MB/s [ 183.3 IOPS] < 5439.00 us>

Profile: Default Test: 1 GiB (x3) [D: 0% (5/1574GiB)] Mode: [Admin] Time: Measure 5 sec / Interval 5 sec Date: 2021/01/26 14:28:39 OS: Windows Server 2012 R2 Server Standard (full installation) [6.3 Build 9600] (x64)

The article about calculating IOPS I read: https://ryanfrantz.com/posts/calculating-disk-iops.html

The information of ThinkSystem 530-8i RAID adapter: https://lenovopress.com/lp0651-thinksystem-raid-530-series-internal-raid-adapters

Jason Yang
  • 21
  • 2
  • What is your stripe size? – shodanshok Jan 26 '21 at 07:41
  • @shodanshok Thank you for replying,it is 64k now. – Jason Yang Jan 26 '21 at 08:04
  • @shodanshok And I found that the "drive cache" option is set to "Disabled",should I enable it?The RAID adapter doesn't support cache,so I wonder if I should enable the drive cache. – Jason Yang Jan 26 '21 at 08:40
  • I'll add that RAID5 with 4 disks is really bad for write performance. Especially without cache and with a 64k stripe size. You really want your file system's block size to be at least as big as how big the across-all-data-disks stripe size is. So in this case you want a ***small*** stripe size - "bigger must be faster" doesn't work with writing to RAID5/6 arrays. You *might* be able to get acceptable write performance with a smaller stripe size on this system. Or as @shodanshok answered, use RAID10 mirroring. – Andrew Henle Jan 26 '21 at 14:16
  • 1
    Also, calling a single 1 MB write operation in a sequence of multiple sequential IO operations a single IOP, while also calling a single 4 KB write operation in a sequence of ***random*** IO operations a single IOP is more than a bit misleading. – Andrew Henle Jan 26 '21 at 14:23

1 Answers1

4

This is a quite common behavior of memory-less HW RAID controller. Basically, the main issue is that without a temporary, safe DRAM cache, writes can not be coalesced, leading to frequent read/modify/write cycles. This is exacerbated by the RAID card disabling HDD own cache.

I suggest you to either use a proper, powerloss-protected RAID card, or to use plain mirroring.

You can read more here.

shodanshok
  • 47,711
  • 7
  • 111
  • 180
  • I'll add that even a RAID controller with write cache can eventually be overwhelmed and start having to do read-modify-write operations if the misaligned/too-small write load is fast enough or random enough so the write operations can't be coalesced fast enough. Eventually, misaligned/too-small random writes to a RAID5/6 array of spinning disks are going to kill performance no matter how good the controller is. And those are the types of writes file systems tend to make under "general usage" patterns, especially when the RAID stripe size is quite a bit larger than the file system block size – Andrew Henle Jan 26 '21 at 14:28
  • Thank you very much. I had changed to a new RAID controller that cache is supported. – Jason Yang Jan 28 '21 at 04:47
  • What are your results after adding new controller? Just curious. Performance should be better. – Stuka Jan 28 '21 at 09:25
  • @Stuka Yes,it is improved significantly.We change to a 730-8i RAID controller,and the performance is fine as other servers that using same RAID controller. – Jason Yang Jan 29 '21 at 06:48