1

I use a gem called whenever to manage my cron jobs. In cronfile, I have every 1 minute cron job which call a task XXXX. My config/schedule.rb is like this：

every '* * * * *' do
  rake "XXXXXXXX"
end

This cron job is working fine with make slight delay. Task XXXX starts to run its first line a few seconds after process is created. Since this task finishes in less than 1 minute, I should never have multiple processes at the same time.

However, the server is heavily loaded, this delay will become a few minutes.
This leads that many undone processes remain in my process list beacause cron job creates a process every minute.

This will cause the server to become heavier, if worst comes to worst, the server is completely dead.

why does it happen? How can I prevent cronjob from to delay calling a task?

user16012143
  • 139
  • 7
  • 1
    Rake tasks aren't done to run them every minute, as every time you run them, they load the whole rails environment with them. You should change your approach and considering to move that logic to a background job and work on the logic to trigger that job for an already-running rails instance. – Alter Lagos May 24 '21 at 04:46
  • Thank you for your comment. "Rake tasks aren't done to run them every minute" what does it mean? – user16012143 May 24 '21 at 08:00
  • `I have every 1 minute cron job which call a task XXXX` <- that's what I meant, cron job + rake task + 1 minute, that's clearly overloading the server. You should use another approach where you don't need to run a rake task every minute (in other words, start your rails env from scratch every minute), but running the task logic in an already running rails instance. – Alter Lagos May 24 '21 at 10:24
  • I agree, it doesn't sound like the problem is with the whenever Ruby gem or with crontab. The machine is simply overloaded. You might be able to eliminate the thrashing simply by lengthening the period to five minutes. If the command that's taking more than a minute is a Rake task in a Rails app then you might not need to load the `:environment` in that Rake task and that might help it go faster. If none of that helps then I agree that you need a background job system of some kind, like maybe Active Job. – Ryan Porter May 27 '21 at 15:57

1 Answers1

0

you can add a dependent task to check whether the server is running or not, fail fast (a.k.a. fail early), for example i want to verify that my rails server is already started on port 3000 before call the rake :test

# Rakefile
task :check_localhost do
  pid = system("lsof -i tcp:3000 -t")
  fail unless pid # or you can use `abort('message')`
end

task :test => :check_localhost do
  puts "****** THIS IS TEST ******"
end
Lam Phan
  • 3,405
  • 2
  • 9
  • 20
  • Thank you so much. In my case, the server is running but is so heavily loaded that it takes more than minutes to call my rake task. so I think that I cannot detect it by another task. – user16012143 May 24 '21 at 07:04
  • @user16012143 how about synchronize that rake task, so that if the first time the task run and stuck to wait the heavy service, e.g 5 minutes then 4 next later task (emit each 1 minute) will wait until the first take done ? – Lam Phan May 24 '21 at 10:24
  • How a rake task is going to share the same process as rails server? They will always be two separate processes. – Kashif Umair Liaqat May 24 '21 at 21:53
  • reference: https://stackoverflow.com/questions/3983883/how-to-ensure-a-rake-task-only-running-a-process-at-a-time – Lam Phan May 25 '21 at 02:43