0

I have a task on ec2 that will be triggered every time a file gets uploaded to an s3 bucket. It seems like for each file uploaded, an instance will start to run the job. The thing is, our vendor likes to upload a bunch of files at the same time and this triggers a bunch of instances to start which we don't have sufficient CPU for, so some of the files just never get processed.

Is there a way to limit the number of the containers starting? or even better, is there a way to trigger containers per time of upload instead of per number of files?

Kim Luu
  • 91
  • 5
  • 1
    What is your architecture? How do you start ec2 instances based on s3 uploads? – Marcin Jul 22 '21 at 02:01
  • I also noticed that all your questions have answers, yet not a single one accepted. Accepting good answers is not onlly a good practice, but it reduces number of duplicates and increases chances of your issues being actually answered. – Marcin Jul 22 '21 at 02:02
  • @Marcin I did basically base on this link https://medium.com/@bowbaq/trigger-an-ecs-job-when-an-s3-upload-completes-3559c44c37d1 – Kim Luu Jul 22 '21 at 17:39

0 Answers0