0

I have 3 parts to my application

  • API Server (API Gateway)
  • Frontend
  • Lambda

I have setup CodePipeline to build my application stack.

I have CodeBuild to build my API gateway SAM application. And CloudFormation to deploy. This CloudFormation exports a variable ApiEndpoint. This variable is used my my web CodeBuild to use.

enter image description here

The problem now is CloudFormation is saying

No export named skynet-infra-server-ApiEndpoint found. Rollback requested by user.

I understand now that its because the Server build has not complete thus that variable is not exported. But how can I resolve this? 

AWSTemplateFormatVersion : '2010-09-09'
Description: 'Skynet stack for Infrastructure. CodePipeline, S3'

Parameters:
  PipelineName:
    Type: String
    Description: Pipeline Name (Lower case only, since S3 bucket names can only have lowercase)
    Default: skynet-pipeline
  AppName:
    Type: String
    Description: App Name
    Default: skynet
  GitHubOwner:
    Type: String
    Description: GitHub Owner
    Default: 2359media
  GitHubRepo:
    Type: String
    Description: GitHub Repo
    Default: 'skynet'
  GitHubBranch:
    Type: String
    Description: GitHub Branch
    Default: master
  GitHubToken:
    Type: String
    Description: GitHub Token
    NoEcho: true

Resources:
  S3WebBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub '${AppName}-web'
      WebsiteConfiguration:
        IndexDocument: index.html
        RoutingRules:
          - RedirectRule:
              ReplaceKeyPrefixWith: '#'
            RoutingRuleCondition:
              HttpErrorCodeReturnedEquals: '404'

      AccessControl: PublicRead
      Tags:
        - Key: Cost Center
          Value: !Ref AppName
        - Key: Owner
          Value: Jiew Meng

  Pipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      Name: !Ref PipelineName
      RoleArn: !GetAtt [PipelineRole, Arn]
      ArtifactStore:
        Location: !Ref PipelineArtifactStore
        Type: S3
      DisableInboundStageTransitions: []
      Stages:
        - Name: GitHubSource
          Actions:
          - Name: Source
            ActionTypeId:
              Category: Source
              Owner: ThirdParty
              Version: 1
              Provider: GitHub
            Configuration:
              Owner: !Ref GitHubOwner
              Repo: !Ref GitHubRepo
              Branch: !Ref GitHubBranch
              OAuthToken: !Ref GitHubToken
            OutputArtifacts:
              - Name: SourceCode
        - Name: Build
          Actions:
          - Name: Lambda
            InputArtifacts:
              - Name: SourceCode
            OutputArtifacts:
              - Name: LambdaPackage
            ActionTypeId:
              Category: Build
              Owner: AWS
              Version: 1
              Provider: CodeBuild
            Configuration:
              ProjectName: !Ref CodeBuildLambda
          - Name: Server
            InputArtifacts:
              - Name: SourceCode
            OutputArtifacts:
              - Name: ServerPackage
            ActionTypeId:
              Category: Build
              Owner: AWS
              Version: 1
              Provider: CodeBuild
            Configuration:
              ProjectName: !Ref CodeBuildServer

        - Name: CreateChangeSet
          Actions:
          - Name: Lambda
            InputArtifacts:
              - Name: LambdaPackage
            OutputArtifacts:
              - Name: LambdaDeployment
            ActionTypeId:
              Category: Deploy
              Owner: AWS
              Version: 1
              Provider: CloudFormation
            Configuration:
              ActionMode: CHANGE_SET_REPLACE
              ChangeSetName: !Sub
                - '${PipelineName}-lambda'
                - {PipelineName: !Ref PipelineName}
              RoleArn: !GetAtt [CloudFormationRole, Arn]
              StackName: !Sub
                - '${PipelineName}-lambda'
                - {PipelineName: !Ref PipelineName}
              TemplatePath: 'LambdaPackage::SkynetLambdaPackaged.yml'
              Capabilities: CAPABILITY_NAMED_IAM
              ParameterOverrides: !Sub '{"AppName": "${PipelineName}-lambda"}'
          - Name: Server
            InputArtifacts:
              - Name: ServerPackage
            OutputArtifacts:
              - Name: ServerDeployment
            ActionTypeId:
              Category: Deploy
              Owner: AWS
              Version: 1
              Provider: CloudFormation
            Configuration:
              ActionMode: CHANGE_SET_REPLACE
              ChangeSetName: !Sub
                - '${PipelineName}-server'
                - {PipelineName: !Ref PipelineName}
              RoleArn: !GetAtt [CloudFormationRole, Arn]
              StackName: !Sub
                - '${PipelineName}-server'
                - {PipelineName: !Ref PipelineName}
              TemplatePath: 'ServerPackage::SkynetServerPackaged.yml'
              Capabilities: CAPABILITY_NAMED_IAM
              ParameterOverrides: !Sub '{"AppName": "${PipelineName}-server"}'

        - Name: Deploy
          Actions:
          - Name: Lambda
            ActionTypeId:
              Category: Deploy
              Owner: AWS
              Version: 1
              Provider: CloudFormation
            Configuration:
              ActionMode: CHANGE_SET_EXECUTE
              ChangeSetName: !Sub
                - '${PipelineName}-lambda'
                - {PipelineName: !Ref PipelineName}
              StackName: !Sub
                - '${PipelineName}-lambda'
                - {PipelineName: !Ref PipelineName}
          - Name: Server
            ActionTypeId:
              Category: Deploy
              Owner: AWS
              Version: 1
              Provider: CloudFormation
            Configuration:
              ActionMode: CHANGE_SET_EXECUTE
              ChangeSetName: !Sub
                - '${PipelineName}-server'
                - {PipelineName: !Ref PipelineName}
              StackName: !Sub
                - '${PipelineName}-server'
                - {PipelineName: !Ref PipelineName}
          - Name: Web
            InputArtifacts:
              - Name: SourceCode
            ActionTypeId:
              Category: Build
              Owner: AWS
              Version: 1
              Provider: CodeBuild
            Configuration:
              ProjectName: !Ref CodeBuildWeb

  CodeBuildLambda:
    Type: AWS::CodeBuild::Project
    Properties:
      Name: !Sub '${PipelineName}-lambda'
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/nodejs:7.0.0
        Type: LINUX_CONTAINER
        EnvironmentVariables:
          - Name: S3_BUCKET
            Value: !Ref PipelineArtifactStore
      ServiceRole: !Ref CodeBuildRole
      Source:
        BuildSpec: 'lambda/buildspec.yml'
        Type: CODEPIPELINE

  CodeBuildServer:
    Type: AWS::CodeBuild::Project
    Properties:
      Name: !Sub '${PipelineName}-server'
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/nodejs:7.0.0
        Type: LINUX_CONTAINER
        EnvironmentVariables:
          - Name: S3_BUCKET
            Value: !Ref PipelineArtifactStore
      ServiceRole: !Ref CodeBuildRole
      Source:
        BuildSpec: 'server/buildspec.yml'
        Type: CODEPIPELINE

  CodeBuildWeb:
    Type: AWS::CodeBuild::Project
    Properties:
      Name: !Sub '${PipelineName}-web'
      Artifacts:
        Type: CODEPIPELINE
      Environment:
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/nodejs:7.0.0
        Type: LINUX_CONTAINER
        EnvironmentVariables:
          - Name: S3_BUCKET
            Value: !Ref S3WebBucket
          - Name: API_URL
            Value:
              Fn::ImportValue:
                !Sub '${PipelineName}-server-ApiEndpoint'
      ServiceRole: !Ref CodeBuildRole
      Source:
        BuildSpec: 'web/buildspec.yml'
        Type: CODEPIPELINE

  PipelineArtifactStore:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub '${PipelineName}-artifacts'
      VersioningConfiguration:
        Status: Enabled

  CodeBuildRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub '${PipelineName}-codebuild'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          Effect: Allow
          Principal:
            Service: codebuild.amazonaws.com
          Action: sts:AssumeRole
      Policies:
        - PolicyName: !Sub '${PipelineName}-codebuild'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Resource: 'arn:aws:logs:*:*:*'
                Action:
                - 'logs:CreateLogGroup'
                - 'logs:CreateLogStream'
                - 'logs:PutLogEvents'
              - Effect: Allow
                Resource:
                  - !Sub 'arn:aws:s3:::codepipeline-${AWS::Region}-*/*'
                  - !Sub
                    - '${PipelineArtifactStoreArn}/*'
                    - {PipelineArtifactStoreArn: !GetAtt [PipelineArtifactStore, Arn]}
                Action:
                  - 's3:GetObject'
                  - 's3:GetObjectVersion'
                  - 's3:PutObject'
              - Effect: Allow
                Resource:
                  - !Sub
                    - '${S3WebArn}/*'
                    - {S3WebArn: !GetAtt S3WebBucket.Arn}
                Action:
                  - 's3:*'

  CloudFormationRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub '${PipelineName}-cloudformation'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Effect: Allow
          Principal:
            Service: cloudformation.amazonaws.com
          Action:
          - sts:AssumeRole
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/AWSLambdaExecute'
      Policies:
        - PolicyName: !Sub '${PipelineName}-cloudformation'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Resource: '*'
                Action:
                - 's3:GetObject'
                - 's3:GetObjectVersion'
                - 's3:GetBucketVersioning'
              - Effect: Allow
                Resource: 'arn:aws:s3:::codepipeline*'
                Action:
                - 's3:PutObject'
              - Effect: Allow
                Resource: '*'
                Action:
                - 'lambda:*'
              - Effect: Allow
                Resource: !Sub 'arn:aws:apigateway:${AWS::Region}::*'
                Action:
                - 'apigateway:*'
              - Effect: Allow
                Resource: '*'
                Action:
                - 'lambda:CreateEventSourceMapping'
                - 'lambda:DeleteEventSourceMapping'
                - 'lambda:GetEventSourceMapping'
              - Effect: Allow
                Resource: '*'
                Action:
                - 'iam:GetRole'
                - 'iam:CreateRole'
                - 'iam:DeleteRole'
                - 'iam:PassRole'
                - 'iam:AttachRolePolicy'
                - 'iam:DetachRolePolicy'
                - 'iam:DeleteRolePolicy'
                - 'iam:PutRolePolicy'
              - Effect: Allow
                Resource: '*'
                Action:
                - 'iam:PassRole'
              - Effect: Allow
                Resource: !Sub 'arn:aws:cloudformation:${AWS::Region}:aws:transform/Serverless-2016-10-31'
                Action:
                - 'cloudformation:CreateChangeSet'
              - Effect: Allow
                Resource: '*'
                Action:
                - 'events:*'

  PipelineRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub '${PipelineName}-pipeline'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Action: ['sts:AssumeRole']
          Effect: Allow
          Principal:
            Service: [codepipeline.amazonaws.com]
      Path: /
      Policies:
        - PolicyName: SkynetPipeline
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Action:
                - 's3:GetObject'
                - 's3:GetObjectVersion'
                - 's3:GetBucketVersioning'
                Effect: 'Allow'
                Resource: '*'
              - Action:
                - 's3:PutObject'
                Effect: 'Allow'
                Resource:
                - !GetAtt [PipelineArtifactStore, Arn]
              - Action:
                - 'codecommit:CancelUploadArchive'
                - 'codecommit:GetBranch'
                - 'codecommit:GetCommit'
                - 'codecommit:GetUploadArchiveStatus'
                - 'codecommit:UploadArchive'
                Effect: 'Allow'
                Resource: '*'
              - Action:
                - 'codedeploy:CreateDeployment'
                - 'codedeploy:GetApplicationRevision'
                - 'codedeploy:GetDeployment'
                - 'codedeploy:GetDeploymentConfig'
                - 'codedeploy:RegisterApplicationRevision'
                Effect: 'Allow'
                Resource: '*'
              - Action:
                - 'elasticbeanstalk:*'
                - 'ec2:*'
                - 'elasticloadbalancing:*'
                - 'autoscaling:*'
                - 'cloudwatch:*'
                - 's3:*'
                - 'sns:*'
                - 'cloudformation:*'
                - 'rds:*'
                - 'sqs:*'
                - 'ecs:*'
                - 'iam:PassRole'
                Effect: 'Allow'
                Resource: '*'
              - Action:
                - 'lambda:InvokeFunction'
                - 'lambda:ListFunctions'
                Effect: 'Allow'
                Resource: '*'
              - Action:
                - 'opsworks:CreateDeployment'
                - 'opsworks:DescribeApps'
                - 'opsworks:DescribeCommands'
                - 'opsworks:DescribeDeployments'
                - 'opsworks:DescribeInstances'
                - 'opsworks:DescribeStacks'
                - 'opsworks:UpdateApp'
                - 'opsworks:UpdateStack'
                Effect: 'Allow'
                Resource: '*'
              - Action:
                - 'cloudformation:CreateStack'
                - 'cloudformation:DeleteStack'
                - 'cloudformation:DescribeStacks'
                - 'cloudformation:UpdateStack'
                - 'cloudformation:CreateChangeSet'
                - 'cloudformation:DeleteChangeSet'
                - 'cloudformation:DescribeChangeSet'
                - 'cloudformation:ExecuteChangeSet'
                - 'cloudformation:SetStackPolicy'
                - 'cloudformation:ValidateTemplate'
                - 'iam:PassRole'
                Effect: 'Allow'
                Resource: '*'
              - Action:
                - 'codebuild:BatchGetBuilds'
                - 'codebuild:StartBuild'
                Effect: 'Allow'
                Resource: '*'

Outputs:
  WebsiteUrl:
    Description: Website URL
    Value: !GetAtt S3WebBucket.WebsiteURL
Jiew Meng
  • 84,767
  • 185
  • 495
  • 805
  • A step in `CodeBuildWeb` could fetch the `API_URL` using the AWS API. – Laurent Jalbert Simard Sep 20 '17 at 14:15
  • @LaurentJalbertSimard, I guess thats a good idea ... but it seems alittle hackish – Jiew Meng Sep 21 '17 at 14:53
  • After years of using CloudFormation, there's a lot of time I wish there was a clean way to do everything but that's unrealistic. As long as your workaround doesn't break the ideas behind IaC (automation, replicability, immutability, etc), I no longer call them "hacks". – Laurent Jalbert Simard Sep 21 '17 at 15:02

1 Answers1

0

Here's two approaches that should work:

  1. You could put the Web CodeBuild action after the Server CloudFormation update, and then pass the output from the CloudFormation action as input to CodeBuild. Seeing as CodePipeline only allows one input to the CodeBuild action you may need to put all your build logic inside an overridden buildspec rather than having it inside the artifact.

  2. Call the CloudFormation APIs to look up these values as part of the build.

TimB
  • 1,457
  • 8
  • 10