AWS Subnet is out of IP Addresses (using CDK)

Question

I've been trying de deploy a RestApiGateway with 20 lambda functions since a while now, and I keep getting the error message Subnet is out of IP addresses every time I try to deploy. Any idea why this may be happening?

Here's how my Vpc stack looks like:

        this.vpc = new ec2.Vpc(this, 'vpc', {
        cidr: '10.0.0.0/21',
        maxAzs: this.azs,
        natGateways: this.azs,
        subnetConfiguration: [
            {
                subnetType: ec2.SubnetType.PUBLIC,
                name: `${APP_PREFIX}-public-subnet`,
                cidrMask: 24,
            },
            {
                subnetType: ec2.SubnetType.PRIVATE,
                name: `${APP_PREFIX}-private-subnet`,
                cidrMask: 28,
            },
        ],
    });

...and here's the gateway:

const interfaceVpcEndpoint = vpc.addInterfaceEndpoint(`${APP_PREFIX}-endpoint`, {
    service: ec2.InterfaceVpcEndpointAwsService.APIGATEWAY,
    subnets: { subnetType: ec2.SubnetType.PRIVATE },
});

const restApiLogGroup = new logs.LogGroup(stack, `${APP_PREFIX}-log-group`, {
    logGroupName: `${APP_PREFIX}-log-group`,
});

return new gateway.RestApi(stack, `${APP_PREFIX}-rest-api`, {
    restApiName: `${APP_PREFIX}-rest-api`,
    endpointConfiguration: {
        types: [gateway.EndpointType.PRIVATE],
        vpcEndpoints: [interfaceVpcEndpoint],
    },
    retainDeployments: true,
    deployOptions: {
        stageName: 'v2',
        loggingLevel: gateway.MethodLoggingLevel.INFO,
        dataTraceEnabled: true,
        accessLogDestination: new gateway.LogGroupLogDestination(restApiLogGroup),
        accessLogFormat: gateway.AccessLogFormat.jsonWithStandardFields(),
    },
    policy: new iam.PolicyDocument({
        statements: [
            new iam.PolicyStatement({
                effect: iam.Effect.DENY,
                principals: [new iam.AnyPrincipal()],
                actions: ['execute-api:Invoke'],
                resources: ['execute-api:/*/POST/*'],
                conditions: {
                    StringNotEquals: {
                        'aws:sourceVpc': vpc.vpcId,
                    },
                },
            }),
            new iam.PolicyStatement({
                effect: iam.Effect.ALLOW,
                principals: [new iam.AnyPrincipal()],
                actions: ['execute-api:Invoke'],
                resources: ['execute-api:/*/POST/*'],
            }),
        ],
    }),
});

The idea is that the lambda functions should not be accessible from the internet, only from a Fargate service that I will deploy via another stack.

What else is running in that subnet? The error message isn't lying to you. — Parsifal, Mar 08 '21 at 17:16
This is a new account, and those are the only two stacks that I run. Are you referring to some resource in particular? — Sammy, Mar 08 '21 at 17:17
Are some of the IP addresses in your private subnet being used by some other services? You'll get only 11 IPs in your private net. Chances are all of them are used up, thus you getting this error message. Check the IP allocations of your private subnet. — Shweta, Mar 08 '21 at 17:19
Thank you. How come I only get 11 IPs, where do you get this number from? Also, the failure happens indeed when I deploy more than 10 lambda functions as per the stack above. Does this mean that each lambda function is taking up an IP address? Why so? Aren't they all exposed behind the same Gateway? — Sammy, Mar 08 '21 at 17:20
Not very sure about how lambdas work behind a gateway. Probably @Parsifal's answer can help you on that but regd the availability of IP addresses is simple. If you take 28 mask, you get a count of 16 available addresses out of which first and last are reserved for broadcasting and stuff and the other 3 are reserved by Amazon. This happens with every CIDR you create. Aws will always reserve 5 and give you the remaining for use. — Shweta, Mar 08 '21 at 17:32

Parsifal · Accepted Answer · 2021-03-08T17:32:25.033

3

I think this is the problem:

cidrMask: 28

A /28 network allows 16 IP addresses, and AWS takes 3 of those addresses for itself.

I'm not familiar with the CDK classes to create a subnet, so can't say whether the overall approach that you're using is correct. I'm assuming that it's smart enough to break the VPC CIDR into subnet blocks based on what you provide for a mask.

I recommend giving at least a /24 allocation to each subnet; I prefer a /20. And a /16 to the VPC. Network configuration is free when you start, extremely expensive to correct later.

edited Mar 08 '21 at 17:32

answered Mar 08 '21 at 17:22

Parsifal

3,928
5
9

Thank you, I can do this, but I'm quite worried so as to why each lambda function that I attach to my Gateway may be taking an IP address up for itself?! – Sammy Mar 08 '21 at 17:24
3

@Sammy - in some cases multiple Lambdas can share the same network interface (IP address) when running inside a VPC. In others they can't. For example, if you have a different security group attached to each. – Parsifal Mar 08 '21 at 17:26
Thank you, that ended up helping me find the solution. – Sammy Mar 09 '21 at 07:44

score 0 · Answer 2 · answered Mar 09 '21 at 07:43

0

I ended up explicitly adding the default security group created by the VPC Construct to each Lambda and that worked.

answered Mar 09 '21 at 07:43

Sammy

3,395
7
49
95

AWS Subnet is out of IP Addresses (using CDK)

2 Answers2