1

I am looking to find some good patterns and anti-patterns for deploying mirrored environments (for simplicity lets say an EC2 Instance, and RDS and S3 bucket, which is a pretty common set-up). Lets say we have to do this hundreds or even thousands of times. I've batted some ideas around such as

  • Multiple accounts - Single purpose - Use all Regions

    • We deploy one instance of a VPC per region, and deploy our set of services in that region.
    • Good, Guarantees isolation and no noisy neighbors, TF modules or CloudFormation Templates won't be complicated
    • Bad, a friggin management nightmare

  • Single Account - Multipurpose

    • We slice up our VPC into multiple subnets and deploy resources per subnet grouping
    • Good, Easier to manage, more for less
    • Bad, You are soft limited to 20 subnets per region (16 regions * 20), possibility of noisy neighbors, networking could end up being spaghetti

I'm looking at more ways to do this and why they'd be bad (technical debt, unmaintainable) or good (easily reusable and etc)

Thanks a million

ehime
  • 597
  • 2
  • 7
  • 15

1 Answers1

3

So a few general points to say on this quite complex topic - it depends a lot on what you are really trying to achieve.

You have three options:

  1. Single VPC - single set of large subnets - in your example that would be 4 - 2 'public' subnets and 2 'private' subnets. Then use security groups to isolate 'deployments' - there is no benefit to using subnets for separation that I see, other than then having to manage the IP address space and lots of subnets. Ultimately the only difference between a subnet is typically: AZ/route-table/nacl/dhcp-options - only use a new subnet if one of those changes. A subnet does not give you any 'noisy neighbour' issues itself. It is not a layer 2 domain in a classic 'vlan' sense and the upstream internet gateway is horiztonally scalable with no limits, as per: Amazon VPC FAQs

  2. Multiple VPC - if you had a single account - you can have multiple VPCs in a region, the soft limit is 5 VPCs, but the hard max is:

    The number of VPCs in the region multiplied by the number of security groups per VPC cannot exceed 10000.

    Amazon VPC Limits

    Which is pretty high, in your example you could say there might be 4 security groups (ELB, EC2 ASG, RDS, Admin access) so that means 2,500 VPCs theoretically? I've not heard of anyone have that, but it could be an option.

    However, another thing to think about depending on how auto-scalable your platform is, are some limits are account wide and if you hit them for one deployment, then it might affect another - e.g. simply instance count of a particular type - or Lambda concurrent execution limits. So that leads to the 3rd option...

  3. Multiple accounts - Now you can create new accounts via API thanks to the AWS Organizations API, however the Limits page is unfortunately vague on what that limit is in terms of number of accounts. Though I have heard of larger enterprises having 1000's of accounts. See: Limits of AWS Organizations and How to Use AWS Organizations to Automate End-to-End Account Creation

Overall for your use case, you'll want to have a clean CloudFormation stack which can be fully re-used with minimal variation - simply for consistency of deployment, operations and support. To me this points to either VPC as the unit of deployment or Account as the unit of deployment. You will need to be careful to watch for limits in both cases. Doing it within a VPC, either with large subnets, or split by subnet, will ultimately become messy to maintain - my subjective view.

Alex Moore
  • 1,704
  • 5
  • 12