0

I have a small 32bit-ARM-based board with three disk attached via USB 2.0. This has been running fine with ZFS 0.7.1 and 0.6.x for a long time, and I do not mind the reduced disk performance. (I am using ZFS elsewhere, I like to have this compatible and available when the real servers go down). However, I had to upgrade the distribution, 0.7.1 does not install nicely anymore, 0.8.5/6 is not stable under load, but 2.0.1 is completely stable again (features are still 0.7.1 compatible). I am not using ZVOLs on this device, just regular datasets.

My problem: The CPU load gets very high on writes. I am under the impression that the slow disks cause the CPU load and iowaits to increase massively causing other tasks on the CPU to get interrupted too. I had a look at this guide which addresses latency, but that does not solve my problem.

Which ZFS parameters should I address, if I know my theoratical maximum write rates, to avoid high CPU-loads?

Martin
  • 326
  • 2
  • 14
  • 1
    How are your disks configured? As three separate disks or as a RAID pool? – Massimo Jan 21 '21 at 23:22
  • @Massimo Raid-Z1, with compression, no de-dup. datasets with maximum efficiency/minimal features/options. ARC at 256MB, no ZIL, no L2ARC. – Martin Jan 22 '21 at 00:24
  • PS: ARC at 256MB, no ZIL, no L2ARC. The excessive load occurs with just a single client writing big files. – Martin Jan 22 '21 at 00:29
  • 1
    "Raid-Z1, with compression"... this seems indeed a good recipe for high CPU load on writes. – Massimo Jan 22 '21 at 08:07
  • @Massimo Remember, the disks are attached over a very slow connection, so compression is useful. And I do not think that it is a limiting factor: As a quick benchmark, I ran pigz (parallel gzip) which should be only slightly more efficient than lz4 and the throughput is a multiple of the actual write speeds.And as I said, it worked well on old kernels and zfs versions. – Martin Jan 23 '21 at 14:05

1 Answers1

0

Short answer: you can't.

Long answer: zfs uses certain amount of CPU to process the iops, most of which are compression and checksuming. If you have the CPU not capable of handling the amount needed - you'll get freezes/spikes. Partial solution is to disable compression and set a weaker/faster checksumming algo. You can also play with disk iops scheduler, but this tends to be more a moral support measure. For academic/testing usage you can also disable ZIL (zfs set sync=disabled <dataset>), but for production use this is generally a bad idea.

drookie
  • 8,625
  • 1
  • 19
  • 29