13

I am using Sidekiq for job management with a Rails APP. I would like to know how do you get the arguments of the failed jobs (Ids, Objects, error message, etc...)? On the WebUI, all you get is the nr of failed jobs. And if I understand correctly, the default is to add up all the times a given job failed. I have deployed my app and it is running on several workers. It is difficult to have to go over every single worker and try to find out, especially when you have a sidekiq.log that is many days old.

I search for an answer here and on the web. One of the closest was described on the folowing links.

How to find failed jobs list in sidekiq?

This allows to find out how many failed jobs I have over a period of time. However, I still do not know how to know exaclty what job failed and why.

Basically, I would like to somehow collect the job_ids and check periodically which ones have failed or if there is an easier way, just query Sidekiq/Redis and see what are the arguments and errors of the failed jobs.

I also visited this link: Get error message out of Sidekiq job

Here is an example job I am using

class ClassificationJob < ActiveJob::Base
 queue_as :default

 def perform(entity)
  entity.preprocess!
  entity.classify!
 end
end

I tried to modify to this

class ClassificationJob < ActiveJob::Base
 queue_as :default

 def perform(entity)
  entity.preprocess!
  entity.classify!
  store entity_id: entity.id.to_s

  entity_id = retrieve :entity_id
 end
end

But it spits the following error:

ArgumentError: You cannot include Sidekiq::Worker in an ActiveJob: ClassificationJob

Thanks,

Yannick

Community
  • 1
  • 1
Yann
  • 151
  • 1
  • 1
  • 8

2 Answers2

16

You are looking for retries. A "retry" is Sidekiq's term for a job which has failed and will be retried sometime in the future.

The Web UI uses the Sidekiq API documented on the API wiki page to list retries on the Retries tab:

https://github.com/mperham/sidekiq/wiki/API#retries

retries = Sidekiq::RetrySet.new
retries.each do |job|
  p [job.klass, job.args, job.jid]
end
Mike Perham
  • 21,300
  • 6
  • 59
  • 61
  • Thank you for the reply. From my understanding, there are jobs that have failed and will be retried. On the webUI, they are listed when one clicks on the "Retries" button. Others are failed (there is not way to click). It seems to me that some of these are not even retried (otherwise, it would take more time to get on the failed list because of they have to be retried many times). My special interest is on those ones. Am I misinterpreting anything here?. – Yann Nov 08 '15 at 17:00
  • 2
    A "failure" is a job execution attempt that raised an error. A job can fail N times, that's why failures is unclickable: it's just a counter. You are right that if a job disables retries, it is ephemeral and will not show in the UI except to increment failures. – Mike Perham Nov 08 '15 at 21:10
  • Thank your for your kind and fast replies Mike. – Yann Nov 08 '15 at 23:18
  • Just another question, Mike. If the counter adds up all the failures (followed by a retry or not), then the number of failures should be always larger that the number of retries. Right? This is however almost never the case for my app. Or I might ask again. Is the "Failure" counter incremented only when a job has been tried for the max number of retries? – Yann Nov 09 '15 at 00:08
  • Failures should be >= retries. – Mike Perham Nov 09 '15 at 04:32
  • Unless the code is fixed (for whatever reason the retry failed in the first time) and then they all processed without failure. In that case, Failures would not be greater than retries. – courtsimas Sep 10 '19 at 04:13
5

I've just been searching for a way to narrow down different types of failures in the console from a given queue.

After a bit of digging,

To get a list of your workers:

query = Sidekiq::Failures::FailureSet.new.map(&:item)

query.map { |item| item['class'] }.uniq # Gives you a sense of what failed
query.map { |item| item['wrapped'] }.uniq # Gives you the job's original class
query.map { |item| item['queue'] }.uniq # Gives you a name of each of the queues
query.map { |item| item['args'] }.uniq # This would give you all the argument hashes for all failures.

The args hash has the following keys: 

job_class
job_id
queue_name
arguments
locale

I think you have to have this gem for it to work: https://github.com/mhfs/sidekiq-failures

peoplespete
  • 1,104
  • 1
  • 11
  • 15
  • 6
    When I tried this in console. I get this NameError: uninitialized constant Sidekiq::Failures error. – Surya Nov 20 '18 at 13:54