0

I am in middle of running a sidekiq job and let's say for some reason an error occurs. For that particular error we don't want to update the retry_count of the sidekiq job but want to trigger the retry. Is there any particular way to do it?.

I tried deleting one of the job and modifying the items to not update retry queue and push it again. However it causes inconsistency as when sidekiq realises there was a error the deleted job comes up with updated retry count.

I am doing all this in middleware as there is where sidekiq properties are accessible.

def call(worker, item, queue)
   begin
      job = get_job_from_sidekiq(item.queue)
      # say some error occurs
   rescue HandleThisError
     job["retry_count"] = [msg["retry_count"].to_i - 1,0].max
   end
end

Basically avoiding the retry count to increase. This doesn't seem to be working do we have any work around for it?

Amit Singh
  • 135
  • 1
  • 1
  • 8
  • Is it not working because the `job["retry_count"]` doesn't actually change, or because you are only setting it in memory and never saving it to Redis? – Chiperific Aug 13 '22 at 17:39
  • I guess it doesnt change because my changes are in memory and never saving it up in redis manually. However I was changing in middleware so was thinking if I change `item["retry_count"]` or job it should have updated, guess i was wrong. The context does get update but actual retry_count never gets changed. I tried to do job.delete and push new job with new count but that doesn't work as well for some reason job doesn't get deleted not sure why. Sorry little bit new to sidekiq. Also i would like to avoid redis manually for this if possible. – Amit Singh Aug 13 '22 at 17:50

1 Answers1

1

Sidekiq gives you global and worker level options for setting retry behavior. The best on-the-rails approach would be to work within these options.

Editing the details of a specific job will require updating those details in Redis, which is doable manually if you know how, but that's really what Sidekiq is doing for you and personally, I'm not sure I'd take this approach.

Based on your comment, the only way I can think of to accomplish this without touching Redis would be to intervene just before Sidekiq writes job details back to Redis for the next retry. You could change the retry_count value just before Sidekiq writes it to Redis. You'd probably be hacking the gem, and probably during requeue.

Note that Sidekiq also has a bulk_requeue, so there are at least two ways your job could be sent to Redis for storage.

Chiperific
  • 4,428
  • 3
  • 21
  • 41
  • Hi @chiperific Thanks for the above comment actually i tried to do some similar stuff however in my case I don't need requeue instead something I need to follow with retry only so that if other errors come they can be catched up from there only. Howevert thanks to your comment I did similar stuff and trying to hijack retry_job classes function. Any way needed to update redis that's the only way out I could think of and also to avoid the duplication I had to consume this error so it doesn't raise Exception and creates and entry for retry. – Amit Singh Aug 13 '22 at 21:28