5

I would like to create a Managed Compute Environment for AWS Batch, but use EC2 User Data to configure the instances as they are brought into the ECS fleet that Batch is scheduling jobs onto.

It shouldn't matter, but the purpose of the User Data script is to pull down large data files onto an InstanceStore that the Docker containers will reference.

This is possible in ECS, but I have found no way to pass User Data to a Managed Batch Compute Environment.

At most, I can specify the AMI. But since we're going with Managed, we must use the Amazon ECS-optimized AMI.

I'd prefer to use EC2 User Data as the solution, as it gives a entry-point for any other bootstrapping we wish to perform. But I'm open to other hacks or solutions, so long as they are applicable to a Managed Compute Environment.

lukeo05
  • 1,833
  • 2
  • 14
  • 13
mcg256
  • 150
  • 1
  • 6

2 Answers2

2

You can create an AMI based on the AWS provided AMI, and customize it. It will still be managed since the Batch and/or ECS daemon is running on it.

As a side note I’m trying to do the same thing but no luck so far. I may end up creating a custom AMI and include the configure script in the AMI itself in /etc/rc.local. Not ideal but I don’t think Batch can pass a user data script other than what it needs. I am still looking into this.

Ryan
  • 36
  • 2
  • I've marked this answer as correct since this is also what our AWS solutions architect suggested, and it worked. – mcg256 Apr 04 '19 at 19:35
0

You can create a launch template containing your user-data. Then assign this launch template to your compute environment. Keep in mind that you might have to clean the cloud init directory in your AMI since it probably was already spun up once (at ami creation). Launch template userguide

user11284211
  • 1
  • This should be the correct answer, although at the time of writing the question @Ryan's answer was a work-around that we used. The Launch template method does not require one to re-process their AMI's, can take variables in CloudFormation templates, lends itself better to automation, etc. – mcg256 Dec 11 '19 at 19:47