0

I'm trying to secure a pipeline for analyzing controlled-access genomic data with Amazon Elastic MapReduce (EMR), and it would help to know the minimal set of outbound rules required of the master and slave security groups of an EMR cluster. I'm sure it differs from region to region, and the IP ranges given at http://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html probably subsume them, but it would be great to know exactly which CIDR blocks we should worry about. It looks like EMR pokes just the right holes among the inbound rules for everything to work, but I've found the cluster gets stuck on provisioning if the outbound rules are anything other than "allow all traffic."

verve
  • 775
  • 1
  • 9
  • 21

1 Answers1

2

We had the identical problem. The way we addressed this problem is by doing the following.

From the ip-ranges.json, use the EC2 CIDR block & AMAZON service cidr block. You may substract CLOUDFRONT & ROUTE53 blocks.

The reason is you need to be able to talk to EMR webservice endpoints that are hsoted outside your VPC. EMR uses a subset of EC2 instances to spin up cluster.

If you have a support contract, ask Amazon to provide you with the CIDR block (we paid for a consulting engagement and this was one of the things they did).

Also, as the EMR webservice is on a public DNS endpoint (not 10.*), there should be a route to the internet gateway.

user1452132
  • 1,758
  • 11
  • 21
  • Can you elaborate on what Amazon support contributed? You say the information I need is in ip-ranges.json, but there's a _bunch_ of CIDR blocks that would have to be included among security group outbound rules for a given region. Are you saying Amazon support narrowed the CIDR blocks you had to consider? – verve Nov 12 '15 at 14:06
  • Yes, they reduced the ip ranges by identifying a subset of 'AMAZON' service ranges. For e.g. they know the ip ranges for EMR, so we dont need to include all 'AMAZON' blocks. The alternative is the 'bunch'. – user1452132 Nov 12 '15 at 16:27