I am running 6 sidekiq jobs launched from a rake task at the same time. Each task is making active record connections and sending some data over sftp.
When the task runs, the CPU usages jumps to 100%, and drops down after some time.
All my tasks log start, but no done or fail logging is observed.
I see the problem happening with only my worker, so I am assuming the problem is related to my worker/task.
Specs:
ruby '2.6.5'
rails '6.0'
sidekiq '5.2.5'
RAILS_MAX_THREAD = 16 Sidekiq concurrency = 3
config.sidekiq_store = ConnectionPool.new(size: [RAILS_MAX_THREAD, 25].max, timeout: 5)
What is the suggested approach in these cases? Also please let me know if more info is required.
I have used sidekiq-limit_fetch, to set concurrency for my specific queue to 1. But the behaviour remains same.
