4

What is a good way to Reduce the number of workers on a machine in Python-RQ?

According to the documentation, I need to send a SIGINT or SIGTERM command to one of the worker processes on the machine:

Taking down workers

If, at any time, the worker receives SIGINT (via Ctrl+C) or SIGTERM (via kill), the worker wait until the currently running task is finished, stop the work loop and gracefully register its own death.

If, during this takedown phase, SIGINT or SIGTERM is received again,the worker will forcefully terminate the child process (sending it SIGKILL), but will still try to register its own death.

This seems to imply a lot of coding overhead:

  • Would need to keep track of the PID for the worker process
  • Would need to have a way to send a SIGINT command from a remote machine

Do I really need to custom build this, or is there a way to do this easily using the Python-RQ library or some other existing library?

Community
  • 1
  • 1
Chris Dutrow
  • 48,402
  • 65
  • 188
  • 258
  • I'm not aware there is a generic solution for this, and most projects I know about implement a custom solution tailored to their peculiar needs. So unless Python-RQ provides something out of the box (not familiar with it), you may need to custom build it. It does not sound like so much work. – Pascal Belloncle Feb 28 '13 at 09:22
  • 1
    For 1, you may need to track the PID, and rq provides it https://github.com/nvie/rq/blob/master/rq/worker.py#L166 – woozyking Mar 01 '13 at 15:13

1 Answers1

6
  1. Get all running workers using rq.Worker.all()
  2. Select the worker you want to kill
  3. Use os.kill(worker.pid, signal.SIGINT)
gioi
  • 1,463
  • 13
  • 16