workling doesnt work when the database is down in my rails app

Question

i want to schedule my worker using a cron job to check for database connection periodically (say like 5 mins) and update a memcache key accordingly. so in my app if i find the memcache variable to be set. i render my pages differently then, when the database is up.

But the problem is, the worker doent start when the database is down. when the database is up. it correctly finds out that the database connection is present and update the memcache variable and everything works fine.

I dont know, why worker doesnt start when the database is down. Am running out on a deadline. any help very much appreciated !

Update:

This is the error i get when the workling doesnt start

/apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/activerecord-2.1.1/lib/active_record/connection_adapters/mysql_adapter.rb:527:in real_connect': Can't connect to MySQL server on '10.223.2.50' (111) (Mysql::Error) from /apps/Symantec/shasta/website/vendor/plugins/workling/script/../lib/workling/starling/poller.rb:35:injoin' from /apps/Symantec/shasta/website/vendor/plugins/workling/script/../lib/workling/starling/poller.rb:35:in listen' from /apps/Symantec/shasta/website/vendor/plugins/workling/script/../lib/workling/starling/poller.rb:35:ineach' from /apps/Symantec/shasta/website/vendor/plugins/workling/script/../lib/workling/starling/poller.rb:35:in listen' from /apps/Symantec/shasta/website/vendor/plugins/workling/script/listen.rb:19 from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/application.rb:203:inload' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/application.rb:203:in start_load' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/application.rb:296:instart' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:51:in watch' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:51:infork' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:51:in watch' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:45:ineach' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:45:in watch' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:44:inloop' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:44:in watch' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:84:instart_with_pidfile' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:64:in fork' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:64:instart_with_pidfile' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/monitor.rb:111:in start' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/application_group.rb:149:increate_monitor' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/application.rb:283:in start' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/controller.rb:70:inrun' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons.rb:143:in run' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/cmdline.rb:112:incall' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons/cmdline.rb:112:in catch_exceptions' from /apps/Symantec/shasta/website/install/local/ruby-1.8.7-p299/lib/ruby/gems/1.8/gems/daemons-1.1.0/lib/daemons.rb:142:inrun' from script/workling_starling_client:17

Arsen7 · Answer 1 · 2010-11-26T14:07:30.317

Maybe the worker tries to connect to the database when starting (always) and throws an exception? Have you any errors logged by the worker?

Did you write your worker in Rails? Maybe write a shell script, which will assume the database is down when the worker cannot start?

UPDATE: In your stack trace there is the starting point: script/workling_starling_client:17. What is there, in the line 17?

As the first line (the exception message itself) says that "real_connect': Can't connect to MySQL server on '10.223.2.50' (111) (Mysql::Error)" then it will be enough if you wrap the line 17 (possibly with a few more) in a "rescue" block, and check the error message whether it has the answer you are looking for:

(Of course, don't stop here. Continue your checks, as the lack of exception does not mean that the connection IS established)

begin
  line_17_is_here
rescue => e
  if e.message =~ /Can't connect to MySQL/
    handle_your_no_connection_state
  else
    raise e
  end
end

The question is: can you handle the no-connection state without ActiveRecord?

i am thinking of dropping workling altogether for this. but instead run a background ruby scirpt inside scripts/check.rb which updates the memcache variable which my website checks to see if db is connected or not. but in my ruby script , if i use require 'memcache' . it throws an and loadError .cant find memcache. — anusuya, Nov 26 '10 at 17:26

workling doesnt work when the database is down in my rails app

1 Answers1