1

I have a list of directories that I'd like to delete. However when I do them all at one, load spikes on the box. Does anyone have an example of a script that would wait until load was under a certain level before proceeding with the deletion (interation of the loop)

ckliborn
  • 2,778
  • 4
  • 25
  • 37

2 Answers2

2

Better still, use ionice.

ionice -c3 ./deletion-script

This'll cause your delete command only to run when the disk IO is free, so it has the lowest priority. Cyberciti has a nice little article on all the variants you have, and how to use them.

Jay
  • 6,544
  • 25
  • 34
  • 1
    Tried it - still caused load to spike. Its a very busy box. – ckliborn Sep 27 '12 at 21:40
  • Do you have cPanel? cPanel has a binary called `cpuwatch` designed for exactly this, but I haven't stumbled across a free equivalent. – Jay Sep 27 '12 at 21:43
  • 1
    You should check `cpulimit` daemon (http://cpulimit.sourceforge.net) – lik Sep 27 '12 at 21:47
  • It sometimes helps to use both "nice" and "ionice" on top of each other. Depends a bit of the exact flavor of Unix and the specifics of the system you are running, but if it works in your case it would be a quick and dirty easy solution. – Tonny Sep 27 '12 at 21:50
  • @lik `cpuwatch` allows you to specify a load average values, which is a little different to `cpulimit` which is more about cpu time. – Jay Sep 27 '12 at 21:54
  • @Jay your are correct, `cpuwatch` tool is more appropriate in this case, but unfortunately it`s a part of paid product, so simple copy of `/usr/local/cpanel/bin/cpuwatch` binary executable may violate license. – lik Sep 27 '12 at 22:54
2

ionice is probably the best/simplest solution, but its only really delaying the inevitable and potentially just mean prolonged medium load versus a short run of high load.

There is a good write up here http://www.depesz.com/2010/04/04/how-to-remove-backups/ on a systematic controlled approach.

Changing the io scheduler, removing journals, atime and diratime are also contributory factors. There is also a different deletion binary fastrm , you can find the man here http://linux.die.net/man/1/fastrm

Don't try this at home

A while ago, we were testing faster deletion methods of deletion for terabytes of data. In the end it actually proved quicker to start a rm or mv then interrupt the process with ^c. Then restart the machine with a forced fsck - which then would clean up the inodes and free up the subsequent space on the file system.

It worked surprisingly well and far quicker than a traditional delete - but I would never ever do it on a production system. Ever.

Community
  • 1
Ben Lessani
  • 5,244
  • 17
  • 37
  • 1
    Note: `ionice` only works with the default [CFQ scheduler](http://en.wikipedia.org/wiki/CFQ). – ewwhite Sep 27 '12 at 21:51