1

I have a backend written in Adonis v4.1 (node.js) that I'm deploying on AWS ECS. Inside the Task Definition of my API I have Adonis and Redis, linked by bridge network.

I'm using adonis-scheduler to run a cronjob every 30 minutes. This cronjob makes a db query and for each row it creates a job (with adonis-queue-pro library). The job concurrency is 1 (so if I have 30 records will be processed one job at a time). Each job makes and external API call and updates the record in the db.

On localhost everything works fine, but on staging environment sometimes happens that my API Task has more than one instance and more than one cronjob starts at the same exact time. So instead of having on job for each record I have multiple jobs for each record that runs simultaneously. This is a big problem.

Is there a way to handle this situation and have only the cronjob of one instance to execute? Or maybe could I have a Task Definition with only the scheduler part in one single instance (always) and the API and redis with many instances?

The problem is that the code I wrote is deeply linked to my backend, so I cannot use something like lambda functions or other external services.

Agilulfo
  • 101
  • 3
  • 10

1 Answers1

0

With ECS, you can have REPLICA or DAEMON service types. DAEMON ensures that only one instance of the task is placed on each instance in the cluster. In your case, it'd mean having exactly one instance behind the cluster. If this constraint is acceptable, give it a go and see if the problem persists.

peter n
  • 1,210
  • 13
  • 18
  • Thank you for answering. I'm trying to follow your approach: I created a task definition only for my scheduler and set it to `DAEMON`. But it shows me `Desired count 2 (Automatic)`, so as default it set 2 instances as desired number. During the service creation I cannot edit this field. I want it to always be 1 active task, and when deploying I want to first let the active task stop and then start the new one. How could I achieve this? – Agilulfo Sep 14 '20 at 09:08
  • How many EC2 instances do you have behind the cluster? This forces the number of tasks started. – peter n Sep 14 '20 at 17:57