3

Is there a way to run a process (e.g. ls, du, find .. etc) within a strictly limited amount of resources (i.e. cpu, memory, I/O) aside from using nice & ionice, as I didn't find them effective enough in my case.

I have a production server, that has directories with ginormous amount of files, that need to be removed, moved, get their total size .. etc

Zoredache
  • 130,897
  • 41
  • 276
  • 420
yalu
  • 31
  • 2
  • What are you trying to do? – Michael Hampton Mar 13 '13 at 22:02
  • I am trying to run du & rm on directories that contain a huge amount of files without increasing the CPU Load – yalu Mar 13 '13 at 22:06
  • What specifically is the problem? Run top at the same time your process is running. Are you out of memory, CPU, or do you have a high iowait percentage? – Zoredache Mar 13 '13 at 22:07
  • 1
    Hitting so many inodes whether through rm or du dramatically increases the cpu load, mainly I have an issue with IO consumption of these processes, and I want them to run in a very slow and graceful way, even if took me day to delete the files, or get directories sizes – yalu Mar 13 '13 at 22:10
  • Use `nice(1)` to decrease the CPU load by the "non important" processes. – vonbrand Mar 13 '13 at 23:24
  • I think even nice would not work as the stat() call is done in kernelspace by the VFS layer (am I right?) – Martino Dino Mar 14 '13 at 00:38

3 Answers3

1

There is one more extreme level of process control than (re)nice: chrt. You may set the process to SCHED_IDLE. In combination with ionice IDLE that should do the job.

There is a really device mapper target which unfortunately didn't make it into the mainline kernel yet which gives you even more control: ioband

Another option would be: Put this process into a VM. Direct VFS access is possible in KVM and you can precisely define how much I/O the VM gets.

Hauke Laging
  • 5,285
  • 2
  • 24
  • 40
0

You can use taskset to limit the processgroup to a specific CPU, and if you nice and ionice it to the max that will help - but it'll still run fast enough to flush your VFS cache (if that's an issue) and the VFS cache is outside the limits quota - but using limits isn't going to help since it just disables new process when they run out.

Really if you want to walk the tree slowly in the background then you're going to have write some code.

symcbean
  • 21,009
  • 1
  • 31
  • 52
0

Please note that ionice usage may differ according to the classes you specify; using a nice value of 19 with an ionice 'idle' class worked good to me in a similar case of yours without increasing the CPU/IO too much

This how you can use it:

nice -n 19 'command'
ionice -c 3 -p `ps aux | grep command | grep -v grep | awk {'print $2'}`
minniux
  • 408
  • 2
  • 6