I'm wondering how to efficiently scale sidekiq on my server.
The author says he uses 6 sidekiq processes, each with concurrency set to 25. He also suggests trying concurrency of 50 if you have heavy I/O.
What's the difference between adding more processes or increasing concurrency? Which is better for helping my app process more jobs faster?
I've tested with concurrency as high as 300 but at that level you start to run into trouble with the MRI. Mike, the author, doesn't recommend concurrency higher than 50 threads per instance of sidekiq.
My understanding is that additional instances will allow your sidekiq jobs to be processed faster as they aren't getting held up by Ruby's MRI. The only downside with additional instances would be a certain amount of memory overhead for each instance.