3

The problem

  • After upgrading NAS system from Buster to Bullseye disks cannot spin down.

My NAS setup

  • 5x mechanical disk WD RED, no RAID, ext4 without journaling
  • Debian Bullseye, kernel 5.10
  • system is loading from sixth SSD disk (not to wake up mechanical ones)
  • hd-idle utility to set spin down after 30 mins of inactivity hd-idle -i 0 -a sda -i 1800 -a sdb -i 1800 -a sdc -i 1800 -a sdd -i 1800 -a sde -i 1800 -l /var/log/hd-idle.log
  • no hardware changes in NAS while upgrading to Bullseye, also no changes in config of hd-idle
  • NAS is in this hardware setup for 8+ years and also software setup, mainly disks were spinning down after 30 mins for years without any problems

What I have tried

  • If I use hd-idle -t sda, disk will immediately spin down, but after waking up (due to some regular disk activity), it never fall asleep again.
  • If disks are forced to spin down with hd-idle -t, they are not waking up without regular activity - so no repeating job is waking them.
  • I was suspicious about smartmontools, if there is some change in Bullseye which is waking them up. I was playing a little with /etc/smartd.conf and -n standby, but no success, so I disabled smartd completely to find if smartmontools are the hidden bad-man, but no luck, looks they aren't.
  • lsof | grep /media gives nothing (media is mountpoint for all those disks)
  • I was playing a little with iotop also with nmon. But nothing interesting found, only regular activity with SSD (system) disk.
  • I cannot run iosnoop for some reason. The packge perf-tools-unstable is installed properly, but calling iosnoop ends with bash: iosnoop: command not found even as a root. I did not find the cause of this.

It is obvious you cannot debug my system, but I will appreciate for any hints how to find the problem, who is preventing disks to spin down :-)

Martin
  • 151
  • 6
  • Since the disks don't spin up without "normal activity" i think it's not activity that's the problem. What kernel and what version of HD-idle are you using? On https://github.com/adelolmo/hd-idle it states: Use disk partitions to calculate activity The disk activity is calculated by watching read/write changes on partition level instead of disk level. This is required for kernels newer than 5.4 LTS, because disk monitoring tools change read/write values on disk level, although there's no real activity on the disk itself. – kruemi Jan 31 '22 at 09:51
  • 1
    Kernel: 5.10.0-11-amd64. I did not realized how to get version of hd-idle as there is no -v or --version parameter, so only by date of binary file I can see that I have version from 2018-08-26 and on github (https://github.com/adelolmo/hd-idle/releases?page=2) I see the oldest version 1.0 from 2018-09-16. Funny. I will try a newer version of hd-idle and come back. – Martin Jan 31 '22 at 17:02
  • Unfortunately, also newest version of hd-idle (1.16) produces the same result as described above. – Martin Jan 31 '22 at 17:50
  • Ok. Have you read the steps listed under "run hd-idle" on https://github.com/adelolmo/hd-idle and checked if everything is ok? – kruemi Feb 01 '22 at 08:38

1 Answers1

2

Solved. At least I know, why my disks are not sleeping. I cannot definitely say, how to trace this problem with some linux tools, I found it accidentaly in some discussion.

Looks like a change in udisks2 service. This service is checking your disks and is responsible for automounting your USB sticks etc. Now the service checks also SMART of disks (don't know why). If the disk is sleeping it will not wake it, but if it is active, udisks2 is checking SMART every 10 minutes. So if the time to sleep is 30 minutes, disk will never fall asleep. Also WD Red disks are often not able to set smaller time to sleep than 10 minutes (it is not healthy for disks to spin down too agressive, so it is reasonable limit).

Some sources:

Workaround for NAS system: you probably do not need udisks2 service at all.

sudo systemctl stop udisks2
sudo systemctl mask udisks2
Martin
  • 151
  • 6