8

I am having the same problem like Rails custom environment Resque.enqueue does not create jobs , but the solution there doesn't work for me.

I'm using Resque for a couple of asynchronous jobs. It works just fine for the staging environment, but for some reason it stopped working on development environment.

For example, if I run the following:

$ rails c development

> Resque.enqueue(MyLovelyJob, 1)

Nothing is enqueued. I check Resque using resque-web

If I run it on staging - it works just fine.

$ rails c staging

> Resque.enqueue(MyLovelyJob, 1)

I have tried to duplicate the 2 environment, and they seem to use absolutely the same configurations (database.yml , config/environment , etc.), but development is still not working.

If I do

 > Resque.enqueue(UpdateInstancesData, 2)
 > => true
 
 > Resque.info
 > => {
 >       :pending => 0,
 >     :processed => 0,
 >        :queues => 0,
 >       :workers => 1,
 >       :working => 0,
 >        :failed => 0,
 >       :servers => [
 >       [0] "redis://127.0.0.1:6379/0"
 >   ],
 >   :environment => "development"
 > }

Any suggestions where to look in order to debug this?

I am running the application via foreman. My Procfile looks like:

faye: rackup faye.ru -s thin -E production

worker1: bundle exec rake resque:work QUEUE=* VERBOSE=1

worker2: bundle exec rake resque:work QUEUE=* VERBOSE=1

clock: bundle exec rake resque:scheduler VERBOSE=1

web: bundle exec rails s

For staging, as mentioned, everything works and the log from foreman is:

17:03:42 clock.1       | 2013-06-26 17:03:42 Reloading Schedule
17:03:42 clock.1       | 2013-06-26 17:03:42 Loading Schedule
17:03:42 clock.1       | 2013-06-26 17:03:42 Scheduling logging_test
17:03:42 clock.1       | 2013-06-26 17:03:42 Schedules Loaded
17:03:43 worker2.1     | *** Starting worker ttttt-mbp.local:69573:*
17:03:43 worker2.1     | *** Registered signals
17:03:43 worker2.1     | *** Running before_first_fork hooks
17:03:43 worker1.1     | *** Starting worker ttttt-mbp.local:69572:*
17:03:43 worker1.1     | *** Registered signals
17:03:43 worker2.1     | *** Checking another_queue
17:03:43 worker2.1     | *** Checking anotherqueue
17:03:43 worker2.1     | *** Checking statused
17:03:43 worker2.1     | *** Found job on statused
17:03:43 worker2.1     | *** got: (Job{statused} | LoggingTest | ["57e89a1c1b24ce6866bcf5d0e1c07f01", {}])

17:06:30 clock.1       | 2013-06-26 17:06:30 queueing LoggingTest (logging_test)
17:06:33 worker1.1     | *** Checking another_queue
17:06:33 worker2.1     | *** Checking another_queue
17:06:33 worker1.1     | *** Checking anotherqueue
17:06:33 worker2.1     | *** Checking anotherqueue
17:06:33 worker1.1     | *** Found job on anotherqueue
17:06:33 worker1.1     | *** got: (Job{anotherqueue} | LoggingTest | ["0d976869a945766e0cfeca83e7349305", {}])
17:06:33 worker1.1     | *** resque-1.24.1: Processing anotherqueue since 1372259193 [LoggingTest]
17:06:33 worker1.1     | *** Running before_fork hooks with [(Job{anotherqueue} | LoggingTest | ["0d976869a945766e0cfeca83e7349305", {}])]
17:06:33 worker1.1     | *** resque-1.24.1: Forked 69955 at 1372259193
17:06:33 worker2.1     | *** resque-1.24.1: Forked 69956 at 1372259193
17:06:33 worker1.1     | *** Running after_fork hooks with [(Job{anotherqueue} | LoggingTest | ["0d976869a945766e0cfeca83e7349305", {}])]
17:06:33 worker1.1     | JOB :: LoggingTest
17:06:33 worker1.1     | 55555
17:06:33 worker1.1     | *** done: (Job{anotherqueue} | LoggingTest | ["0d976869a945766e0cfeca83e7349305", {}])

whereas for development it doesn't seem to enqueue and then find the job. If there is a job already in the queue (pending, left over from staging environment) the workers from development don't process it.

17:01:23 clock.1       | 2013-06-26 17:01:23 Reloading Schedule
17:01:23 clock.1       | 2013-06-26 17:01:23 Loading Schedule
17:01:23 clock.1       | 2013-06-26 17:01:23 Scheduling logging_test
17:01:23 clock.1       | 2013-06-26 17:01:23 Scheduling update_instances_data
17:01:23 clock.1       | 2013-06-26 17:01:23 Schedules Loaded

17:03:10 clock.1       | 2013-06-26 17:03:10 queueing LoggingTest (logging_test)
17:03:14 worker1.1     | *** Checking another_queue
17:03:14 worker2.1     | *** Checking another_queue
17:03:14 worker1.1     | *** Checking anotherqueue
17:03:14 worker2.1     | *** Checking anotherqueue
17:03:14 worker1.1     | *** Checking statused
17:03:14 worker2.1     | *** Checking statused
Community
  • 1
  • 1
Anton Evangelatov
  • 1,397
  • 2
  • 14
  • 31
  • Are the jobs running immediately or not at all? – Elad Meidar Jun 25 '13 at 16:08
  • 1
    The jobs are not enqueued at all, even though you can see the return value "true". This is for development. For staging, the counter at Resque.info changes, and I can see the job using resque-web as well. Obviously something is very wrong with the "development" environment. – Anton Evangelatov Jun 25 '13 at 19:53
  • @antonevangelatov did you started resque work in development like `rake resque:worker ` or `rake resque:work` and if yes have you specified the Queue name to it – Viren Jun 26 '13 at 08:36
  • My foreman Procfile looks like: `faye: rackup faye.ru -s thin -E production` ; `worker: bundle exec rake resque:work QUEUE=*` ; `clock: bundle exec rake resque:scheduler` ; `web: bundle exec rails s` – Anton Evangelatov Jun 26 '13 at 10:49
  • I start foreman with `foreman start -e .env.staging` or `foreman start -e .env.development` . For staging it works perfectly, for development - it doesn't... Inside the .env files I have the following: `RAILS_ENV="staging"` and other environment variables, such as AWS credentials, etc. – Anton Evangelatov Jun 26 '13 at 11:10
  • I am starting to think that some classes are not loaded at the correct time when in development environment. My jobs are inside `app/jobs/*.rb` and they are using classes from `app/lib/*.rb`... it might be that no error message is displayed after `13:41:00 clock.1 | 2013-06-26 13:41:00 queueing LoggingTest (logging_test)` – Anton Evangelatov Jun 26 '13 at 11:42
  • It turns out that not only the jobs are not enqueued, but also the worker is not taking/processing any jobs when in development. Basically I started foreman for staging, it was processing jobs for a while, and then I stopped it. At this point, one job was still pending. When I started foreman for development, I can see that the job is still pending, and it is not taken by the worker. At the same time I see the message "0 of 1 workers working" which means that there is a worker..... – Anton Evangelatov Jun 26 '13 at 12:13
  • @antonevangelatov ok but do you see the message in the queue that your worker is polling too – Viren Jun 27 '13 at 06:42
  • 1
    @Viren - no, I don't. The problem is solved, I don't know if you've noticed my answer below. Basically I had the 'resque_spec' in the development group which is making a double of Resque... then the clock is scheduling a job, which is never executed, since Resque is doubled... this is how I explain it. As soon as I removed 'resque_spec' from the development group, everything was working just fine. – Anton Evangelatov Jun 27 '13 at 10:00

2 Answers2

26

The problem turned out to be that I had the gem 'resque_spec' also for the :development group in my Gemfile. Obviously a very wrong thing to do...

Anton Evangelatov
  • 1,397
  • 2
  • 14
  • 31
0

My problem was that there were orphan workers running in the background. They were picking up jobs instead.

By restarting redis all workers die.

Juan Ibiapina
  • 300
  • 2
  • 7