2

I've decided to start playing with AWS ECS service, and created cluster and one service my issue is that I want to connect it to the AWS auto scaling group. I have followed the following guide.

The guide works, my issue is that its a total waste of money.

The guide says that I need to add machine when the total amount of CPU units that my services reserve is above 75, but in reality my services always reserve 100% because I don't want waste money, also its pretty useless to put 3 nodejs tasks on 2 cpu machine, there is no hard-limit anyway.

I am breaking my head on it for few days now, I have no idea how to make them work together properly

EDIT: Currently this is what happens:

  1. CPU getting above 75%, Service scaling is created 2 new tasks on the same server, which means that now I have 1 instance with 4 tasks

  2. Instance reservation is now 100%, Auto Scaling Group is creating new instance

  3. Once the new instance is created, Service Scaling is removing 2 tasks from the old instance and adding 2 new tasks to the new instance

Its just me or this whole process looks like waste of time? is this how it really should be or (probably) i done something wrong?

Rot-man
  • 18,045
  • 12
  • 118
  • 124
roy
  • 585
  • 5
  • 14

2 Answers2

1

I think you are missing a few insights.

For ECS autoscaling to work properly you also have to set up scaling on a ECS Service level.

Then, the scaling flow would look like this:

  1. ECS Service is reaching 100% used CPU and has 100% reserved CPU
  2. ECS Service scales by starting an additional task, making the reserved CPU total at 200%
  3. Auto scaling group sees there is more Reserved capacity than available capacity and launches a new machine.

In addition, you can perfectly run multiple nodejes tasks on a 2 CPU machine. Especially in a micro service environment, these nodejs services can be quite small (128 CPU for example) and still run perfectly fine all together on the same host machine.

ThomasVdBerge
  • 7,483
  • 4
  • 44
  • 62
  • i dont understand, ECS service cant reach 200% reserved, you get error that you dont have enough CPU to scale and the reserved metric is staying on 100% not 200% – roy May 29 '18 at 11:46
  • Then it will add a new EC2 instance by using the autoscaling group – ThomasVdBerge May 29 '18 at 11:47
  • yea thats what i dont want, i want to be able to reserve 100% of the cluster and i want to create new machine only when i need 200% ex: i have 2 CPU machine, i launch 1 service with 2 tasks of nodejs app that each take 1024 cpu units, if i do it currently, it will scale me up with no reason – roy May 29 '18 at 11:50
  • You can perfectly reserve 100% and scale on EC2 when you need 200%. That's exactly what is posted in my answer. You will need to add scaling on ECS Service and on the underlying EC2 instances – ThomasVdBerge May 29 '18 at 11:52
  • You say that when the service scaling alert trigger (> 75 cpu) the cluster reserved cpu metric will be 200% and not 100%? even tought it didnt created tasks? – roy May 29 '18 at 11:53
  • No, when it creates a new task it would reach 200% reserved if your service has been set up to use 100% reserved – ThomasVdBerge May 29 '18 at 11:54
  • So at the auto scaling group i need to add policy to increase size if cluster reserved > 100? – roy May 29 '18 at 12:00
  • Exactly :-). Two scaling policies: Add one instance when CPUReservation = 100 and Remove one instance when CPUReservation = 50 – ThomasVdBerge May 29 '18 at 12:04
  • 1
    checking, if it works i personally gonna come to your home and hug you – roy May 29 '18 at 12:05
  • i might got it wrong, still getting the same issue as before in the services events i get this - was unable to place a task because no container instance met all of its requirements and the auto scalling group alarm isnt triggering (CPUReservation >= 100 for 1 datapoints within 1 minute) – roy May 29 '18 at 12:14
  • ok i created everything from scratch, now it half works - it does create new instance but it creates it because i used 100% cpu reservation not 200% – roy May 29 '18 at 12:35
1

Eventually, I figured, what I want to do is not possible. It is not possible to create a resource-optimized cluster with ECS like in Kubernetes. (unless of course if you write some magic with lambda)

Service auto-scaling and auto-scaling groups don't work together, you can, however, make it work perfectly with fargate but its expansive, the main issue is that you don't have a trigger to cluster reservation above 100%

MLavoie
  • 9,671
  • 41
  • 36
  • 56
roy
  • 585
  • 5
  • 14