I have batches of 500 messages. And to send them I am using the external API which allows sending only 1 message at a time. Also, they have a rate limit of 10/seconds.
If there is a single instance then I can handle the rate limit by adding a delay between the API calls. But in my case number of instance are not fixed. It totally depends upon the traffic I receive.
Let's say I have 10 instances running and they are processing messages in batches of 500 messages. So, I have to process 5000 messages in total. But the rate limit is 10/seconds so if all the instance invokes the same API with the same credentials then the rate limit get exceeded after a single API call of every instance.
When It will try to send the second message then they will get the 429 error as the rate limit is exceeded.
Now, I want to make sure that combining all the 10 instances will only send the 10 messages per second. How can I implement it?
Any better suggestions or recommendations are appreciated!!
