0

I run a ruby on rails site that processes email - the email is dumped directly into the web app via a POST from postfix.

At times I can get a burst of email coming in causing a prolonged surge in CPU usage making my VPS provider understandable unhappy with me. These emails don't need to be processed in a timely manner - they just need to be (eventually) processed.

Obviously I can't just nice the process as that only looks at the cpu usage on my VPS and can't take into account the cpu usage on the other VPS's.

I have found a utility called cpulimit that will you put hard limits on cpu usage for a particular process. (eg 20%). This seems ideal for this purpose, but I can't work out to integrate with apache/passenger.

Passenger starts up a ruby process for each server and restarts them periodically. Each time the pid will change. Cpulimit needs to be given a pid number for it to act on.

Anyone got any ideas how I could get passenger to fire off a call this command when it's starting up this particular virtual host?

Dave Smylie
  • 141
  • 1
  • 7
  • Flip it around. Have postfix store the email as normal, then have a cron job fire off and read emails, and process them as needed. – becomingwisest Apr 13 '12 at 22:26
  • hmmm. that may work. I don't think I would use cron but could certainly have a separate process reading and feeding in emails at a certain rate (eg 1 every few seconds). I will try this if my cpulimit script below doesn't prove practical – Dave Smylie Apr 14 '12 at 04:03

1 Answers1

0

I'm not sure if this is the ideal solution, but for lack of any better idea, I wrote a short script to monitor this and call cpulimit for me.

I've included it below incase it's of use to anyone else.

#!/home/dgs/.rvm/rubies/ruby-1.9.3-p125-perf/bin/ruby

# script to check for mail server processes and enforce a cpulimit on them

# cmd to get the pid of the process you want to limit. 
YOU WOULD NEED TO MAKE THIS RELEVANT TO YOU
CMD='ps -ef | grep mailserver | grep Rack | grep -v grep | awk "{print \$2}"'

# how often to check    
PERIOD = 60

#array to hold list of the currently limitied processes
limiting = []

while true
  processes  = `#{CMD}`.split("\n")

  # iterate thru the newly found processes, limiting any that
  # aren't already being limited
  processes.each do |p|
   if ! limiting.include? p
     system "cpulimit -p #{p} -l 10 -z &"  
     limiting << p
   end 
  end

   # check if any old processes have been killed
   limiting.each do |p|
     begin
       Process.getpgid( p.to_i )
     rescue Errno::ESRCH
       limiting.delete p
     end
   end
   sleep PERIOD
end
Dave Smylie
  • 141
  • 1
  • 7