0

is there any way to stop consumers from eating low priority messages until all high priority messages are consumed?

I am thinking of checking the queue state in low priority message handler, but this solution doesn't seem elegant and can be danger.

mcek
  • 480
  • 7
  • 17
  • Does setting [worker priority](https://symfony.com/doc/current/messenger.html#prioritized-transports) not work? "_The worker will always **first look** for messages waiting on **async_priority_high**. **If there are none, then** it will consume messages from async_priority_low_". – msg Oct 23 '20 at 03:59
  • 1
    The proposed solution starts consuming low-priority messages just after the last high-priority one starts, even if it didn't finished yet. I think he wants the low messages start processing only when the queue has no high messages in progress (I'm also looking for a solution for that). – Mochilo Nov 20 '20 at 16:39
  • @Mochilo This is exactly what I want to achive. Please give me know if you find anything could help. – mcek Nov 22 '20 at 11:38

1 Answers1

1

Maybe too late, but we found a way that works reasonably well for our needs.

Basically, the high priority job writes a lock in DB. Then, when the low priority job starts, it first checks the lock: if it is there, we reschedule the job with a delay stamp.

The lock is removed as part of the high priority job, as a last step. Then, when the low priority job arrives again, it can be processed. By doing this we are able to manage three queues with different priority levels. However, we gained in complexity in our flows.

Mochilo
  • 301
  • 1
  • 10