6

We run one consumer per python app and run multiple processes based on the queue load. As a next step, I would like to auto scale these consumer processes so that we can avoid unnecessary processes when the queue size is low and add additional processes when the pending messages grow.

How do people handle such an auto scaling mechanism for consumers, if anyone does?

Thanks.

vrtx54234
  • 2,196
  • 3
  • 30
  • 53
  • We are using spring framework's rabbittemplate and it handles auto scaling stuff but it does that by changing number of consumer of course not the number of processes. You can look into it for details. – cool Nov 22 '15 at 11:50
  • @cool thanks for the comment; I hate threads/greenlets in python (very error prone if not used carefully), that's why am looking for a solution that scales processes. Thanks. – vrtx54234 Nov 22 '15 at 16:37
  • @vrtx54234 did you find the solution? – Shashank Madan Jan 20 '21 at 10:33
  • @ShashankMadan no – vrtx54234 Jan 20 '21 at 17:16

0 Answers0