Stack Overflow
Stack Overflow

Questions tagged [sidekiq]

Sidekiq is a background processing framework for Ruby.

Sidekiq is a multi-threaded background job and message processing gem written by Mike Perham and is available on GitHub.

2372 questions
12
votes
1 answer

Debugging sidekiq worker memory leak with jemalloc

So, I've got a memory leak in my Sidekiq worker. I have a worker server with one queue for this worker task only, that gets to about 10G RSS in a week. I tried to reproduce it locally with only 1 worker thread and voila - I get from 200M to 1G in…
Roman
  • 389
  • 1
  • 13
12
votes
1 answer

Sidekiq logging to both terminal and log file

I'm using Sidekiq to queue up some jobs in my Rails server. As per the Logging wiki, it's as simple as adding the following in config/sidekiq.yml --- :verbose: false :pidfile: ./tmp/pids/sidekiq.pid :logfile: ./log/sidekiq.log :concurrency: …
absessive
  • 1,121
  • 5
  • 14
  • 36
12
votes
8 answers

Invoking a large set of SQL from a Rails 4 application

I have a Rails 4 application that I use in conjunction with sidekiq to run asynchronous jobs. One of the jobs I normally run outside of my Rails application is a large set of complex SQL queries that cannot really be modeled by ActiveRecord. The…
randombits
  • 47,058
  • 76
  • 251
  • 433
12
votes
3 answers

Stack level too deep when using carrierwave versions

I'm trying to use a sidekiq worker, which more or less saves a image file to database (using carrierwave). There are few files to save, which are a keyframes extracted from a video file. That's what that worker is about. My image uploader has a few…
mbajur
  • 4,406
  • 5
  • 49
  • 79
12
votes
4 answers

Sidekiq and rails 4 actionmailer never delivers emails

I've setup sidekiq in a rails 4 application for sending emails and processing background jobs. I have also devise which I am using devise_async and a typical contact form emailer. I am using gmail for sending emails. If I remove sidekiq, it sends…
JohnDel
  • 2,092
  • 8
  • 35
  • 64
12
votes
3 answers

Sidekiq deploy to multiple environments

(See below for my detailed config, which is the result of Henley Chiu's answer). I've been trying to wrap my brain around Sidekiq deploys, and I am not really getting it. I have an app with a staging environment, and a production environment, on the…
d3vkit
  • 1,942
  • 1
  • 24
  • 36
12
votes
3 answers

Work with two separate redis instances with sidekiq?

Good afternoon, I have two separate, but related apps. They should both have their own background queues (read: separate Sidekiq & Redis processes). However, I'd like to occasionally be able to push jobs onto app2's queue from app1. From a simple…
Brandon
  • 3,091
  • 2
  • 34
  • 64
12
votes
5 answers

execute only one of many duplicate jobs with sidekiq?

I have a background job that does a map/reduce job on MongoDB. When the user sends in more data to the document, it kicks of the background job that runs on the document. If the user sends in multiple requests, it will kick off multiple background…
Eric Seifert
  • 1,946
  • 2
  • 17
  • 31
11
votes
4 answers

Sidekiq sending jobs to test queue instead of developement queue

I've added specs to cover sidekiq jobs with rspec, but now, when I start a rails server, console, or just sidekiq, I have this warning: WARNING: Sidekiq testing API enabled, but this is not the test environment. Your jobs will not go to Redis. And…
Graham Slick
  • 6,692
  • 9
  • 51
  • 87
11
votes
2 answers

exec sidekiq log on heroku

I want to exec the sidekiq log file on my heroku environment but I cannot find anything in the documentation nor google helps me here. I am sure it has to be sth like exec sidekiq -L log/sidekiq.logbut that command fails on heroku. How can I access…
DonMB
  • 2,550
  • 3
  • 28
  • 59
11
votes
1 answer

How can I test Sidekiq job failures?

A job in sidekiq, upon exception, will be put on the retry queue. Because of that, and because the task is run asynchronously, MyWorker.perform_async(...) can never throw an exception generated in the task code. In testing, however, an exception…
Mark Bolusmjak
  • 23,606
  • 10
  • 74
  • 129
11
votes
3 answers

What's the difference between running rake task in background vs. using gem like Delayed Job, Resque, or Sidekiq?

I need to implement some background processing to 1) send emails, 2) do some API calls. And, whatever system I use, I'll also be combining with some kind of cron scheduler (Whenever likely). I'm curious, I recognize that there's an array of really…
james
  • 3,989
  • 8
  • 47
  • 102
11
votes
3 answers

Errno::EIO: Input/output error -

class FaxFetchWorker include Sidekiq::Worker sidekiq_options :retry => false def perform(job_id=0) logger.warn "perform is invoked." FaxSource.all.each do |source| ... end end end Getting Error Errno::EIO: Input/output…
Dipak Panchal
  • 5,996
  • 4
  • 32
  • 68
11
votes
2 answers

Does Sidekiq execute jobs in the order they are sent to a worker?

I have a rake task which is going to call 4 more rake tasks, in order: rake:one rake:two rake:three rake:four Rake tasks one, two, and three are getting data and adding it to my database. Then rake:four is going to do something with that data. But…
Luigi
  • 5,443
  • 15
  • 54
  • 108
11
votes
1 answer

Log to different logger based on call from Sidekiq or Rails

I have an Rails 3.2 based app that uses Sidekiq 2.12 to run background jobs. The Sidekiq jobs can call the same methods as the interactive Rails app. I would like the methods to log to the Sidekiq log when called from Sidekiq and log to the Rails…
Old Pro
  • 24,624
  • 7
  • 58
  • 106
1 2 3
…
99 100