1

I have recently started working with mesos and marathon. My goal is to have a dynamic cluster. In other words, allow Mesos to start slaves on EC2 on demand, and kill them as they get unused.I hope there is a solution, if Mesos notices that my build cluster is overloaded, it'll start instances and automatically connect them as Mesos slaves. When the load goes down, excessive EC2 instances will be terminated.

DiStephane
  • 105
  • 1
  • 7

1 Answers1

2

It's possible but so far haven't seen it widely used yet, I suppose mainly because the elasticity you seek is in fact delivered by Mesos itself. But if you want to do it, here's how (note that you'd want to run one Mesos agent per EC2 instance):

  • Preferably via an Auto Scaling Group that scales horizontally
  • Alternatively via the Autoscaler, scaling the sizes of your boxes (vertical scaling)
Michael Hausenblas
  • 13,162
  • 4
  • 52
  • 66
  • Thanks for your fast and detailed answer! I saw the links you gave me. But in my case I want to know if there is a possibility to communicate Mesos with AWS via CloudFormation. So when launching an application knowing that the resources available are insufficient, Mesos will communicate with AWS to Create a new Mesos agent (EC2 instance) and if Mesos notices that no application is running on a Mesos agent terminate it – DiStephane Mar 21 '16 at 10:47
  • Short answer is: you'd need to do it yourself with Mesos, ATM. You can use http://mesos.apache.org/documentation/latest/monitoring/ metrics to trigger the launch of a new instance, for example. Once the Mesos Agent is running it will join the cluster and make its resources available. The opposite case is even simpler, Mesos Master will notice that the Agent is not available anymore and make sure its Executors and tasks are launched on other Agents. – Michael Hausenblas Mar 21 '16 at 11:01