1

I've setup a Linux system with RAID-6. Sequential write I/O performance is initially around 600 MiB/s after a reboot, but degrades to between 0.5 MiB/s and 2 MiB/s after the system has been used for a while. I'm using sysbench to benchmark the I/O peformance.

I'm running Gentoo Linux with kernel version 4.19.72. The RAID controller is a Broadcom / LSI MegaRAID Tri-Mode SAS3508 using the megraraid_sas driver. It's configured with 5 Seagate ST600MM0069 drives. RAID strip size is 256 K according to perccli. Drives are formatted with ext4 using the stride=64 and stripe_width=320 options. Writeback and caching are enabled.

This is a new system. I had the same problem with an older previous system. It was also configured with RAID-6, but the system wasn't really being used so the issue was never noticed. It was repurposed a little while ago and the issue was noticed once usage increased.

The older system is using a Broadcom / LSI MegaRAID SAS 1078 RAID controller configured with 6 Western Digital WD1002FAEX-00Z3A drives.

Update

I've reconfigured the server with RAID-5. I also realized that stripe_width=320 was the incorrect value for the number of disks I'm using. I've corrected that to strip_width=384. The issue still remains. It seems to not be specific to RAID. I suspected RAID-6 because we have other, older, systems running RAID-5 that we don't have any problem with.

Robert
  • 23
  • 5
  • 1
    I'm confused - where are you doing RAID at? The way I'm reading your question it seems like you are using both hardware RAID and also software RAID. – Rob Pearson Oct 26 '19 at 22:01
  • Please, provide more detailed info. How many disks, how they are connected, how this looks in the OS (`cat /proc/mdstat`, `lsblk`, `blkid` and so on). And a side note: forget about RAID5, never use it, it is like sitting on a barrel of gunpowder. – Nikita Kipriyanov Oct 27 '19 at 07:29
  • I'm running a system in 32-bit mode due to legacy software. It turns out that Linux can't efficiently manage the amount of memory the server has using PAE: https://flaterco.com/kb/PAE_slowdown.html. I was able to fix the issue by by enabling the VMSPLIT_2G configuration option and limiting the amount of addressable memory to 16G using the `mem=0x4000M` kernel boot parameter. – Robert Oct 28 '19 at 20:19

1 Answers1

0

I'm running the system in 32-bit mode due to legacy software. It turns out that Linux can't efficiently manage the amount of memory the server has using PAE: flaterco.com/kb/PAE_slowdown.html.

I was able to fix the issue by by enabling the VMSPLIT_2G configuration option and limiting the amount of addressable memory to 16G using the mem=0x4000M kernel boot parameter.

Robert
  • 23
  • 5