Is there a dsl for AWS EC2?

Question

I am looking at using Amazon cloud services (EC2, S3 etc) for hosting. I've been looking at the JSON metadata that can be specified to configure various instances and the complexity has me concerned. Is there a dsl that will generate valid JSON metadata and more importantly validate the entries?

Answer 1

Unfortunately, I drew a blank after searching for this recently. I'm using Amazon Web Services CloudFormation (is that the JSON metadata you're talking about?).

There are a couple of issues with CloudFormation JSON files:

1. I'm at well over 1,500 lines and it's impossible to read,
2. You can't express everything the API gives you, notably in the area of Virtual Private Clouds,
3. There are lots of bugs that are taking a long time to fix, Elastic Load Balancers losing HTTPS information, for example.

So I've been using straight-up API calls in Scala using the Java API. It's actually really nice.

The Java API has a flavor of "setters" starting with with that return this so they can be chained. In Scala, you can use these to act like a poor-man's DSL. So you can do things like

val updateRequest = new UpdateAutoScalingGroupRequest()
                    .withAutoScalingGroupName(group.getAutoScalingGroupName)
                    .withAvailabilityZones(subnetAZsOfOurVPC)
                    .withVPCZoneIdentifier(subnetNamesOfOurVPC)

as.updateAutoScalingGroup(updateRequest)

Other things are easy to do in Scala with the Java API. For example, group all your Subnets by VPC in a Map, simply do

val subnetsByVPC = ec2.describeSubnets(new DescribeSubnetsRequest).getSubnets.groupBy(_.getVpcId)

Answer 2

In case anyone is still looking for the AWS CloudFormation DSL - we've been using the Ruby DSL for CloudFormation:

https://github.com/bazaarvoice/cloudformation-ruby-dsl

• This neat project provides a tool to convert your existing CloudFormation template(s) to the Ruby DSL
• It will generate valid JSON output
• Validating Ruby template entries is similar to validating a regular CloudFormation template (see cfn-validate-template)
• Your template becomes Ruby code, so it's easy to have reusable modules (DRY)
• You can define local variables
• You can have comments in your DSL template
• Greatly improved readability
• Greatly reduced DSL template size

The CloudFormation request template body size limits are annoying - we have to upload our large CloudFormation templates to S3, then create/update stacks using their S3 URLs.

Answer 3

There is now, though I haven't used it yet: Coffin a CoffeeScript DSL for CloudFormation.

If you're not talking about CloudFormation, but instead more general API talking, then the nicest interface I've found is AWS' own aws-sdk ruby gem. Unlike the other SDKs that they publish which are quite nicely-done-but-crude make-client/make-request/get-response/look-at-result affairs, the ruby SDK wraps a nicer domain-model over the top, so you interact via collections at a higher abstract level.

It also has quite-nice performance features in that you can cache responses to save on round-trips, if you know you don't need fresh responses.

Answer 4

I have CloudFormation templates upwards of 3000 lines. I have found that putting comments in the JSON helps tremendously!! You just have to strip it out before using it. There is a validator that would validate the template and strip out the comments: http://cloudformation-validation.com/

Answer 5

No, as of Feb 2022, AWS still does not provide any domain-specific language for infrastructure as code. They have nothing similar to Azure's Bicep or Terraform's HCL.

This really surprises me, as I generally think of AWS as being more expensive, but ahead of the curve, compared to other major competitors (Azure and GCP).

However, Cloud Formation now allows both JSON and YAML formats. Slight improvement?? IMHO, not really when you have an entire repo that represents your entire cloud stack. If you're using AWS, use Terraform to manage IaC.