1

In my rails app, I use a worker process to scan 45k database records once in 6 hours and send out mails if certain condition is met. This causes the server CPU/Load to spike when the worker is processing. As a result of which other server request gets a performance hit. I tried using find_in_batch to retrieve 1000 records at a time and do the processing. But the CPU utilization is still at the peak level. No big difference i was able to see. Is there any way to handle this , so the CPU utilization doesn't hit the max limit?

user229044
  • 232,980
  • 40
  • 330
  • 338
anusuya
  • 653
  • 1
  • 9
  • 24

2 Answers2

3

I assume you're invoking the job via cron and script/runner. You might try lowering the priority of process with nice:

nice -n 19 /usr/bin/ruby <path to your app>/script/runner <your script>
user229044
  • 232,980
  • 40
  • 330
  • 338
  • This won't affect CPU load, but it will mean that other processes have a higher priority. Unfortunately database calls are all considered equal so a lot of query activity will have a knock-on effect on any production system attached to it. – tadman Jan 26 '11 at 21:19
  • I am using rufus scheduler to scheduler worker process to run in regular interval – anusuya Jan 26 '11 at 21:19
0

Fiddling with the process priority level using nice is one way to do it, but another is to tell your app to chill out a little bit now and then using the sleep or select command:

while (doing_stuff)
  do_stuff

  # Take a break for 0.2 seconds
  select(nil, nil, nil, 0.2)
end

The select call will block for a brief period of time allowing other tasks on the system to run freely. The higher you set this value the slower your job will run, but the lower the impact on the CPU load level.

tadman
  • 208,517
  • 23
  • 234
  • 262
  • 1
    This is a poor use of resources. Your app might run during a time where there is no contention for the CPU, in which case it needlessly spends 0.2 seconds doing nothing for each record in the database. – user229044 Jan 26 '11 at 21:19
  • You could always expand on this to set the delay time to be variable depending on system load. – tadman Jan 27 '11 at 15:12