0

What is working:

Using the serverless framework:

  • I have configured an AWS VPC
  • I have an Amazon Aurora database configured for my VPC
  • I have an AWS API Gateway lambda that is configured for my VPC
  • When I deploy my lambda, I am able to access it publicly via the AWS generated URL: XXX.execute-api.us-east-1.amazonaws.com/prod/outages
  • In my Lambda,I run a very simple query that proves I can connect to my database.

This all works fine.

What is NOT working:

  • I have registered a domain with AWS/Route 53 and added a cert (e.g. *.foo.com)

  • I use the serverless-domain-manager plugin to make my lambda available via my domain (e.g. api.foo.com/outages resolves to XXX.execute-api.us-east-1.amazonaws.com/prod/outages)

This works fine if my lambda is NOT configured for my VPC

But when my lambda IS configured for my VPC, the custom domain api.foo.com/outages does NOT resolve to XXX.execute-api.us-east-1.amazonaws.com/prod/outages

I other words: I can NOT access api.foo.com/outages publicly.

What I need is:

1 - XXX.execute-api.us-east-1.amazonaws.com/prod/outages is available publicly (this works)

2 - My custom domain, api.foo.com/outages points to the SAME lambda as XXX.execute-api.us-east-1.amazonaws.com/prod/outages (in my VPC) and is available publicly (not working. I get: {"message":"Forbidden"})

virtual-private-cloud.yml

service: virtual-private-cloud

provider:
  name: aws
  region: us-east-1
  stage: ${opt:stage, dev}

custom:
  appVersion: 0.0.0
  VPC_CIDR: 10

resources:
  Resources:
    ServerlessVPC:
      Type: AWS::EC2::VPC
      Properties:
        CidrBlock: ${self:custom.VPC_CIDR}.0.0.0/16
        EnableDnsSupport: true
        EnableDnsHostnames: true
        InstanceTenancy: default
    ServerlessSubnetA:
      DependsOn: ServerlessVPC
      Type: AWS::EC2::Subnet
      Properties:
        VpcId:
          Ref: ServerlessVPC
        AvailabilityZone: ${self:provider.region}a
        CidrBlock: ${self:custom.VPC_CIDR}.0.0.0/24
    ServerlessSubnetB:
      DependsOn: ServerlessVPC
      Type: AWS::EC2::Subnet
      Properties:
        VpcId:
          Ref: ServerlessVPC
        AvailabilityZone: ${self:provider.region}b
        CidrBlock: ${self:custom.VPC_CIDR}.0.1.0/24
    ServerlessSubnetC:
      DependsOn: ServerlessVPC
      Type: AWS::EC2::Subnet
      Properties:
        VpcId:
          Ref: ServerlessVPC
        AvailabilityZone: ${self:provider.region}c
        CidrBlock: ${self:custom.VPC_CIDR}.0.2.0/24

  Outputs:
    VPCDefaultSecurityGroup:
      Value:
        Fn::GetAtt:
          - ServerlessVPC
          - DefaultSecurityGroup
      Export:
        Name: VPCDefaultSecurityGroup-${self:provider.stage}
    SubnetA:
      Description: 'Subnet A.'
      Value: !Ref ServerlessSubnetA
      Export:
        Name: vpc-subnet-A-${self:provider.stage}
    SubnetB:
      Description: 'Subnet B.'
      Value: !Ref ServerlessSubnetB
      Export:
        Name: vpc-subnet-B-${self:provider.stage}
    SubnetC:
      Description: 'Subnet C.'
      Value: !Ref ServerlessSubnetC
      Export:
        Name: vpc-subnet-C-${self:provider.stage}

database-service.yml

service: database-service

provider:
  name: aws
  region: us-east-1
  stage: ${opt:stage, dev}
  environment:
    stage: ${opt:stage, dev}

plugins:
  - serverless-plugin-ifelse

custom:
  appVersion: 0.0.1
  AURORA:
    DB_NAME: database${self:provider.stage}
    USERNAME: ${ssm:/my-db-username~true}
    PASSWORD: ${ssm:/my-db-password~true}
    HOST:
      Fn::GetAtt: [AuroraRDSCluster, Endpoint.Address]
    PORT:
      Fn::GetAtt: [AuroraRDSCluster, Endpoint.Port]
  serverlessIfElse:
    - If: '"${opt:stage}" == "prod"'
      Set:
        resources.Resources.AuroraRDSCluster.Properties.EngineMode: provisioned
      ElseSet:
        resources.Resources.AuroraRDSCluster.Properties.EngineMode: serverless
        resources.Resources.AuroraRDSCluster.Properties.ScalingConfiguration.MinCapacity: 1
        resources.Resources.AuroraRDSCluster.Properties.ScalingConfiguration.MaxCapacity: 4

      ElseExclude:
        - resources.Resources.AuroraRDSInstanceParameter
        - resources.Resources.AuroraRDSInstance

resources:
  Resources:
    AuroraSubnetGroup:
      Type: AWS::RDS::DBSubnetGroup
      Properties:
        DBSubnetGroupDescription: "Aurora Subnet Group"
        SubnetIds:
          - 'Fn::ImportValue': vpc-subnet-A-${self:provider.stage}
          - 'Fn::ImportValue': vpc-subnet-B-${self:provider.stage}
          - 'Fn::ImportValue': vpc-subnet-C-${self:provider.stage}
    AuroraRDSClusterParameter:
      Type: AWS::RDS::DBClusterParameterGroup
      Properties:
        Description: Parameter group for the Serverless Aurora RDS DB.
        Family: aurora5.6
        Parameters:
          character_set_database: "utf32"
    AuroraRDSCluster:
      Type: "AWS::RDS::DBCluster"
      Properties:
        MasterUsername: ${self:custom.AURORA.USERNAME}
        MasterUserPassword: ${self:custom.AURORA.PASSWORD}
        DBSubnetGroupName:
          Ref: AuroraSubnetGroup
        Engine: aurora
        EngineVersion: "5.6.10a"
        DatabaseName: ${self:custom.AURORA.DB_NAME}
        BackupRetentionPeriod: 3
        DBClusterParameterGroupName:
          Ref: AuroraRDSClusterParameter
        VpcSecurityGroupIds:
          - 'Fn::ImportValue': VPCDefaultSecurityGroup-${self:provider.stage}

    # this is needed for non-serverless mode
    AuroraRDSInstanceParameter:
      Type: AWS::RDS::DBParameterGroup
      Properties:
        Description: Parameter group for the Serverless Aurora RDS DB.
        Family: aurora5.6
        Parameters:
          sql_mode: IGNORE_SPACE
          max_connections: 100
          wait_timeout: 900
          interactive_timeout: 900

    # this is needed for non-serverless mode
    AuroraRDSInstance:
      Type: "AWS::RDS::DBInstance"
      Properties:
        DBInstanceClass: db.t2.small
        DBSubnetGroupName:
          Ref: AuroraSubnetGroup
        Engine: aurora
        EngineVersion: "5.6.10a"
        PubliclyAccessible: false
        DBParameterGroupName:
          Ref: AuroraRDSInstanceParameter
        DBClusterIdentifier:
          Ref: AuroraRDSCluster

  Outputs:
    DatabaseName:
      Description: 'Database name.'
      Value: ${self:custom.AURORA.DB_NAME}
      Export:
        Name: DatabaseName-${self:provider.stage}
    DatabaseHost:
      Description: 'Database host.'
      Value: ${self:custom.AURORA.HOST}
      Export:
        Name: DatabaseHost-${self:provider.stage}
    DatabasePort:
      Description: 'Database port.'
      Value: ${self:custom.AURORA.PORT}
      Export:
        Name: DatabasePort-${self:provider.stage}

outage-service.yml

service: outage-service

package:
  individually: true

plugins:
  - serverless-bundle
  - serverless-plugin-ifelse
  - serverless-domain-manager

custom:
  appVersion: 0.0.12
  stage: ${opt:stage}
  domains:
    prod: api.foo.com
    test: test-api.foo.com
    dev: dev-api.foo.com
  customDomain:
    domainName: ${self:custom.domains.${opt:stage}}
    stage: ${opt:stage}
    basePath: outages
    custom.customDomain.certificateName: "*.foo.com"
    custom.customDomain.certificateArn: 'arn:aws:acm:us-east-1:395671985612:certificate/XXXX'
    createRoute53Record: true
  serverlessIfElse:
    - If: '"${opt:stage}" == "prod"'
      Set:
          custom.customDomain.enabled: true
      ElseSet:
          custom.customDomain.enabled: false

provider:
  name: aws
  runtime: nodejs12.x
  stage: ${opt:stage}
  region: us-east-1
  environment:
    databaseName: !ImportValue DatabaseName-${self:provider.stage}
    databaseUsername: ${ssm:/my-db-username~true}
    databasePassword: ${ssm:/my-db-password~true}
    databaseHost: !ImportValue DatabaseHost-${self:provider.stage}
    databasePort: !ImportValue DatabasePort-${self:provider.stage}

functions:
  hello:
    memorySize: 2048
    timeout: 30
    handler: functions/handler.hello
    vpc:
      securityGroupIds:
        - 'Fn::ImportValue': VPCDefaultSecurityGroup-${self:provider.stage}
      subnetIds:
        - 'Fn::ImportValue': vpc-subnet-A-${self:provider.stage}
        - 'Fn::ImportValue': vpc-subnet-B-${self:provider.stage}
        - 'Fn::ImportValue': vpc-subnet-C-${self:provider.stage}
    environment:
      functionName: getTowns
    events:
      - http:
          path: outage
          method: get
          cors:
            origin: '*'
            headers:
              - Content-Type
              - authorization

resources:
  - Outputs:
      ApiGatewayRestApiId:
        Value:
          Ref: ApiGatewayRestApi
        Export:
          Name: ${self:custom.stage}-ApiGatewayRestApiId

      ApiGatewayRestApiRootResourceId:
        Value:
           Fn::GetAtt:
            - ApiGatewayRestApi
            - RootResourceId 
        Export:
          Name: ${self:custom.stage}-ApiGatewayRestApiRootResourceId
Ken Taylor
  • 21
  • 3
  • is it regional or cloudfront custom domain? – omuthu Mar 22 '20 at 18:45
  • I'm a bit new to this, but I believe it is a custom domain. I registered it through AWS Route 53. When I look at the record in Route 53, it says "Global" (Route 53 does not require region selection.) – Ken Taylor Mar 22 '20 at 19:12
  • Posting a little code goes a long way: this question is hard to answer without seeing (at least parts of) your serverless template. Plus, readers like to see what you've tried so far. In the absence of any code, I can still hazard a guess: maybe you need to add `endpointType: regional` to the domain manager plugin config? – Mike Patrick Mar 22 '20 at 21:50
  • Thanks @MikePatrick - I wrestled with that and do agree 100%. There are three serverless.yml files so there is quite a bit to post, but I will do that in order to make the question easier to understand and hopefully answer. Thanks very much for your comment. – Ken Taylor Mar 22 '20 at 21:54
  • `XXX.execute-api.us-east-1.amazonaws.com/my-path` , with `my-path` is `prod` ? – Thanh Nguyen Van Mar 23 '20 at 10:12
  • Very sorry @ThanhNguyenVan - yes, it is: XXX.execute-api.us-east-1.amazonaws.com/prod/outages – Ken Taylor Mar 23 '20 at 10:48

2 Answers2

0

I believe you might have missed to add VPC endpoints for API Gateway.

Ensure you create a VPC interface endpoint for API GW and use it in the API you create. This will allow API requests to lambda running within VPC

References:

  1. https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-private-apis.html#apigateway-private-api-create-interface-vpc-endpoint
  2. https://aws.amazon.com/blogs/compute/introducing-amazon-api-gateway-private-endpoints/

Hope this helps !!

omuthu
  • 5,948
  • 1
  • 27
  • 37
  • Thank you so much for your answer! It Private API seems is related to having an endpoint that is only accessible from within your VPC: "Using Amazon API Gateway, you can create private REST APIs that can only be accessed from your virtual private cloud in Amazon VPC..." This is not what I need. What I need is: 1 - XXX.execute-api.us-east-1.amazonaws.com/my-path is available publicly (this works) 2 - My custom domain, api.foo.com/my-path points to the same lambda as XXX.execute-api.us-east-1.amazonaws.com/my-path and is available publicly Sorry if I was not more clear. – Ken Taylor Mar 23 '20 at 09:38
  • I tried changing the endpoint to private, just to give it a try, but I get the following error: "An error occurred: ApiGatewayRestApi - You cannot change the endpoint of a RestApi to PRIVATE if there is a custom domain using it." -- so, I think that API Gateway Private Endpoints is not what I need. Let me know if you feel I've misunderstood you on this. – Ken Taylor Mar 23 '20 at 09:56
0

SOLUTION: We found a solution, in our case the solution is that you can't use wildcard certificates. So if you make a certificate for the exact API URL it does work. Obviously this is not a nice solution for everyone and everything but when you do have this problem and must deploy you can make it work this way.

Also maybe good to mention we did not have any additional problems with the below mentioned case:

I also wanted to add a post I found that might also have lead to this setup not working even if we do get the network to play nice. But again if you or anyone has a working setup like the above mentioned one let me know how you did it.

TLDR

If anyone wants to understand what was going on with API Gateway, take a look at this thread.

It basically says that API Gateway processes regular URLs (like aaaaaaaaaaaa.execute-api.us-east-1.amazonaws.com) differently than how it processes Custom Domain Name URLs (like api.myservice.com). So when API Gateway forwards your API request to your Lambda Function, your Lambda Function will receive different path values, depending on which type of your URL you used to invoke your API.

source: Understanding AWS API Gateway Custom Domain Names

SDM
  • 564
  • 5
  • 11