3

Greeting all,

I'm looking for a way to deploy my application which contains:

  • API Gateway
  • DynamoDB
  • Lambda Functions
  • An S3 bucket

I looked at CloudFormation and CodeDeploy but I'm unsure how to proceed without EC2...

All the information I find is for EC2, I haven't found any information regarding deploying the app above...

The goal is to have a deployment script that deploys app to an environment automatically with technology from AWS. (Basically duplicating my environment)

Any help would greatly be appreciated.

EDIT: I need to be able to export from one AWS account then import onto another AWS account.

Cheers!

Luc Laverdure
  • 1,398
  • 2
  • 19
  • 36

4 Answers4

4

In order to deploy your CloudFormation stack into a "different" environment, you have to parameterize your CloudFormation stack name and resource names. (You don't have to parameterize the AWS::Serverless::Function function in this example because CloudFormation automatically creates a function name if no function name is specified, but for most other resources it's necessary)

Example CloudFormation template cfn.yml using the Serverless Application Model (SAM):

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Deploys a simple AWS Lambda using different environments.

Parameters:
  Env:
    Type: String
    Description: The environment you're deploying to.

Resources:
  ServerlessFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: nodejs12.x
      CodeUri: ./
      Policies:
        - AWSLambdaBasicExecutionRole

  MyBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub 'my-bucket-name-${Env}'

You can add further resources like a DynamoDB table. The API Gateway is automatically created if you're using SAM and provide an Events section in your AWS::Serverless::Function resource. See also this SAM example code from the serverless-app-examples repository.

Example deploy.sh script:

#!/usr/bin/env bash

LAMBDA_BUCKET="Your-S3-Bucket-Name"
# change this ENV variable depending on the environment you want to deploy
ENV="prd"
STACK_NAME="aws-lambda-cf-environments-${ENV}"

# now package the CloudFormation template which automatically uploads the Lambda function artifacts to S3 -> generated a "packaged" CloudFormation template cfn.packaged.yml
aws cloudformation package --template-file cfn.yml --s3-bucket ${LAMBDA_BUCKET} --output-template-file cfn.packaged.yml

# ... and deploy the packaged CloudFormation template
aws cloudformation deploy --template-file cfn.packaged.yml --stack-name ${STACK_NAME} --capabilities CAPABILITY_IAM --parameter-overrides Env=${ENV}

See the full example code here. Just deploy the script using ./deploy.sh and change the ENV variable.

s.hesse
  • 1,900
  • 10
  • 13
  • I'm going to test this soon, does this work to export files then import into another AWS account? – Luc Laverdure Jan 23 '18 at 21:50
  • Thanks, found this as well: https://docs.aws.amazon.com/lambda/latest/dg/serverless-deploy-wt.html?shortFooter=true#serverless-deploy which could be useful. – Luc Laverdure Jan 24 '18 at 00:33
  • And this for account transfer https://aws.amazon.com/premiumsupport/knowledge-center/account-transfer-s3/ – Luc Laverdure Jan 24 '18 at 01:03
  • @LucLaverdure if your intention was to move your whole stack (i.e. copy all resources from one stack to the other), then my example would not fit. – s.hesse Jan 24 '18 at 07:32
  • Your prod and dev environments can share the same s3 bucket for code storage? – TemporaryFix Feb 18 '19 at 21:51
  • 1
    @Programmatic yes, the lambda bucket can be the same for production and dev. It just contains temporary files and the `cloudformation package` command will hash the artifact names when uploading them, so a collision is not so so likely. – s.hesse Feb 19 '19 at 07:38
1

Based JSON examples.

Lambda function AWS::Lambda::Function

  • This example creates a Lambda function and an IAM Role attached to it.

Language: NodeJs.

"LambdaRole": {
    "Type": "AWS::IAM::Role",
    "Properties": {
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": [
                            "lambda.amazonaws.com"
                        ]
                    },
                    "Action": [
                        "sts:AssumeRole"
                    ]
                }
            ]
        },
        "Policies": [
            {
                "PolicyName": "LambdaSnsNotification",
                "PolicyDocument": {
                    "Version": "2012-10-17",
                    "Statement": [
                        {
                            "Sid": "AllowSnsActions",
                            "Effect": "Allow",
                            "Action": [
                                "sns:Publish",
                                "sns:Subscribe",
                                "sns:Unsubscribe",
                                "sns:DeleteTopic",
                                "sns:CreateTopic"
                            ],
                            "Resource": "*"
                        }
                    ]
                }
            }
        ]
    }
},
"LambdaFunctionMessageSNSTopic": {
    "Type": "AWS::Lambda::Function",
    "Properties": {
        "Description": "Send message to a specific topic that will deliver MSG to a receiver.",
        "Handler": "index.handler",
        "MemorySize": 128,
        "Role": {
            "Fn::GetAtt": [
                "LambdaRole",
                "Arn"
            ]
        },
        "Runtime": "nodejs6.10",
        "Timeout": 60,
        "Environment": {
            "Variables": {
                "sns_topic_arn": ""
            }
        },
        "Code": {
            "ZipFile": {
                "Fn::Join": [
                    "\n",
                    [
                        "var AWS = require('aws-sdk');",                        
                        "};"
                    ]
                ]
            }
        }
    }
}

API Gateway AWS::ApiGateway::RestApi

  • This example creates Role, RestAPI, Usageplan, Keys and permission to execute lambda from a Request method.
"MSGGatewayRestApi": {
    "Type": "AWS::ApiGateway::RestApi",
    "Properties": {
        "Name": "MSG RestApi",
        "Description": "API used for sending MSG",
        "FailOnWarnings": true
    }
},
"MSGGatewayRestApiUsagePlan": {
    "Type": "AWS::ApiGateway::UsagePlan",
    "Properties": {
        "ApiStages": [
            {
                "ApiId": {
                    "Ref": "MSGGatewayRestApi"
                },
                "Stage": {
                    "Ref": "MSGGatewayRestApiStage"
                }
            }
        ],
        "Description": "Usage plan for stage v1",
        "Quota": {
            "Limit": 5000,
            "Period": "MONTH"
        },
        "Throttle": {
            "BurstLimit": 200,
            "RateLimit": 100
        },
        "UsagePlanName": "Usage_plan_for_stage_v1"
    }
},
"RestApiUsagePlanKey": {
    "Type": "AWS::ApiGateway::UsagePlanKey",
    "Properties": {
        "KeyId": {
            "Ref": "MSGApiKey"
        },
        "KeyType": "API_KEY",
        "UsagePlanId": {
            "Ref": "MSGGatewayRestApiUsagePlan"
        }
    }
},
"MSGApiKey": {
    "Type": "AWS::ApiGateway::ApiKey",
    "Properties": {
        "Name": "MSGApiKey",
        "Description": "CloudFormation API Key v1",
        "Enabled": "true",
        "StageKeys": [
            {
                "RestApiId": {
                    "Ref": "MSGGatewayRestApi"
                },
                "StageName": {
                    "Ref": "MSGGatewayRestApiStage"
                }
            }
        ]
    }
},
"MSGGatewayRestApiStage": {
    "DependsOn": [
        "ApiGatewayAccount"
    ],
    "Type": "AWS::ApiGateway::Stage",
    "Properties": {
        "DeploymentId": {
            "Ref": "RestAPIDeployment"
        },
        "MethodSettings": [
            {
                "DataTraceEnabled": true,
                "HttpMethod": "*",
                "LoggingLevel": "INFO",
                "ResourcePath": "/*"
            }
        ],
        "RestApiId": {
            "Ref": "MSGGatewayRestApi"
        },
        "StageName": "v1"
    }
},
"ApiGatewayCloudWatchLogsRole": {
    "Type": "AWS::IAM::Role",
    "Properties": {
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "Service": [
                            "apigateway.amazonaws.com"
                        ]
                    },
                    "Action": [
                        "sts:AssumeRole"
                    ]
                }
            ]
        },
        "Policies": [
            {
                "PolicyName": "ApiGatewayLogsPolicy",
                "PolicyDocument": {
                    "Version": "2012-10-17",
                    "Statement": [
                        {
                            "Effect": "Allow",
                            "Action": [
                                "logs:CreateLogGroup",
                                "logs:CreateLogStream",
                                "logs:DescribeLogGroups",
                                "logs:DescribeLogStreams",
                                "logs:PutLogEvents",
                                "logs:GetLogEvents",
                                "logs:FilterLogEvents"
                            ],
                            "Resource": "*"
                        }
                    ]
                }
            }
        ]
    }
},
"ApiGatewayAccount": {
    "Type": "AWS::ApiGateway::Account",
    "Properties": {
        "CloudWatchRoleArn": {
            "Fn::GetAtt": [
                "ApiGatewayCloudWatchLogsRole",
                "Arn"
            ]
        }
    }
},
"RestAPIDeployment": {
    "Type": "AWS::ApiGateway::Deployment",
    "DependsOn": [
        "MSGGatewayRequest"
    ],
    "Properties": {
        "RestApiId": {
            "Ref": "MSGGatewayRestApi"
        },
        "StageName": "DummyStage"
    }
},
"ApiGatewayMSGResource": {
    "Type": "AWS::ApiGateway::Resource",
    "Properties": {
        "RestApiId": {
            "Ref": "MSGGatewayRestApi"
        },
        "ParentId": {
            "Fn::GetAtt": [
                "MSGGatewayRestApi",
                "RootResourceId"
            ]
        },
        "PathPart": "delivermessage"
    }
},
"MSGGatewayRequest": {
    "DependsOn": "LambdaPermission",
    "Type": "AWS::ApiGateway::Method",
    "Properties": {
        "ApiKeyRequired": true,
        "AuthorizationType": "NONE",
        "HttpMethod": "POST",
        "Integration": {
            "Type": "AWS",
            "IntegrationHttpMethod": "POST",
            "Uri": {
                "Fn::Join": [
                    "",
                    [
                        "arn:aws:apigateway:",
                        {
                            "Ref": "AWS::Region"
                        },
                        ":lambda:path/2015-03-31/functions/",
                        {
                            "Fn::GetAtt": [
                                "LambdaFunctionMessageSNSTopic",
                                "Arn"
                            ]
                        },
                        "/invocations"
                    ]
                ]
            },
            "IntegrationResponses": [
                {
                    "StatusCode": 200
                },
                {
                    "SelectionPattern": "500.*",
                    "StatusCode": 500
                },
                {
                    "SelectionPattern": "412.*",
                    "StatusCode": 412
                }
            ],
            "RequestTemplates": {
                "application/json": ""
            }
        },
        "RequestParameters": {
        },
        "ResourceId": {
            "Ref": "ApiGatewayMSGResource"
        },
        "RestApiId": {
            "Ref": "MSGGatewayRestApi"
        },
        "MethodResponses": [
            {
                "StatusCode": 200
            },
            {
                "StatusCode": 500
            },
            {
                "StatusCode": 412
            }
        ]
    }
},
"LambdaPermission": {
    "Type": "AWS::Lambda::Permission",
    "Properties": {
        "Action": "lambda:invokeFunction",
        "FunctionName": {
            "Fn::GetAtt": [
                "LambdaFunctionMessageSNSTopic",
                "Arn"
            ]
        },
        "Principal": "apigateway.amazonaws.com",
        "SourceArn": {
            "Fn::Join": [
                "",
                [
                    "arn:aws:execute-api:",
                    {
                        "Ref": "AWS::Region"
                    },
                    ":",
                    {
                        "Ref": "AWS::AccountId"
                    },
                    ":",
                    {
                        "Ref": "MSGGatewayRestApi"
                    },
                    "/*"
                ]
            ]
        }
    }
}

DynamoDB AWS::DynamoDB::Table

  • This example creates a DynamoDB table MyCrossConfig and an alarms for it.
"TableMyCrossConfig": {
  "Type": "AWS::DynamoDB::Table",
  "Properties": {
    "TableName": "MyCrossConfig",
    "AttributeDefinitions": [
      {
        "AttributeName": "id",
        "AttributeType": "S"
      }
    ],
    "KeySchema": [
      {
        "AttributeName": "id",
        "KeyType": "HASH"
      }
    ],
    "ProvisionedThroughput": {
      "ReadCapacityUnits": "5",
      "WriteCapacityUnits": "5"
    }
  }
},
"alarmTargetTrackingtableMyCrossConfigProvisionedCapacityLowdfcae8d90ee2487a8e59c7bc0f9f6bd9": {
  "Type": "AWS::CloudWatch::Alarm",
  "Properties": {
    "ActionsEnabled": "true",
    "AlarmDescription": {
      "Fn::Join": [
        "",
        [
          "DO NOT EDIT OR DELETE. For TargetTrackingScaling policy arn:aws:autoscaling:",
          {
            "Ref": "AWS::Region"
          },
          ":",
          {
            "Ref": "AWS::AccountId"
          },
          ":scalingPolicy:7558858e-b58c-455c-be34-6de387a0c6d1:resource/dynamodb/table/MyCrossConfig:policyName/DynamoDBReadCapacityUtilization:table/MyCrossConfig."
        ]
      ]
    },
    "ComparisonOperator": "LessThanThreshold",
    "EvaluationPeriods": "3",
    "MetricName": "ProvisionedReadCapacityUnits",
    "Namespace": "AWS/DynamoDB",
    "Period": "300",
    "Statistic": "Average",
    "Threshold": "5.0",
    "AlarmActions": [
      {
        "Fn::Join": [
          "",
          [
            "arn:aws:autoscaling:",
            {
              "Ref": "AWS::Region"
            },
            ":",
            {
              "Ref": "AWS::AccountId"
            },
            ":scalingPolicy:7558858e-b58c-455c-be34-6de387a0c6d1:resource/dynamodb/table/MyCrossConfig:policyName/DynamoDBReadCapacityUtilization:table/MyCrossConfig"
          ]
        ]
      }
    ],
    "Dimensions": [
      {
        "Name": "TableName",
        "Value": "MyCrossConfig"
      }
    ]
  }
}

s3 bucket AWS::S3::Bucket

  • This example creates a Bucket with name configbucket- + AWS::AccountId 
"ConfigBucket": {
  "Type": "AWS::S3::Bucket",
  "Properties": {
    "BucketName": {
      "Fn::Join": [
        "",
        [
          "configbucket-",
          {
            "Ref": "AWS::AccountId"
          }
        ]
      ]
    }
  },
  "DeletionPolicy": "Delete"
}

Now you need to put altogether, make the reference in the template, Etc.

Hope it helps!

Ele
  • 33,468
  • 7
  • 37
  • 75
0

My guess would be that you could use CloudFormation for such an app, but I'm also unfamiliar.

What I have had success with is writing small scripts which leverage the awscli utility to accomplish this. Additionally, you'll need a strategy for how you setup a new environment.

Typically, what I have done is to use a different suffix on DynamoDB tables and S3 buckets to represent different environments. Lambda + API Gateway have the idea of different versions baked in, so you can support different environments there as well.

For really small projects, I have even setup my Dynamo schema to support many environments within a single table. This is nice for pet or small projects because it's cheaper.

jeff
  • 4,325
  • 16
  • 27
  • I'm pretty sure I need CloudFormation for this, but as explained above, the examples (Wordpress delpoyment for example) all use EC2... – Luc Laverdure Jan 22 '18 at 17:38
  • You do not need a CloudFormation to deploy to AWS. They are a convenience and can be easy to make when run against pre configured EC2 instances as you said. But no, you do not need a CloudFormation file to deploy AWS services. I also get the impression that they are difficult to generate by hand. – jeff Jan 22 '18 at 20:27
0

Built my own SDK for deployments, it's in the making...

https://github.com/LucLaverdure/aws-sdk

You will need to use the following shell scripts within the containers:

export.sh
import.sh

Requirements:

  • AWS CLI
  • Python
  • pip
  • npm
  • jq
Luc Laverdure
  • 1,398
  • 2
  • 19
  • 36