AWS CDK Pipelines: How to access internal stack values in post-stage steps?

Question

I have created a simple architecture using aws_cdk.pipelines. There is one stack that that contains a VPC and an RDS cluster inside of that VPC. The RDS cluster automatically creates a security group for itself. I've also set it to create credentials for itself, which it stores in AWS secrets manager. This all works and deploys perfectly.

The issue is that after the database is created I would like to perform some deploy-time operations on the database, such as schema migrations. Ideally I would use a CodeBuildStep, but I'm open to alternatives. My problem is that those variables that were available in the stack are out of scope at the stage level. How can I access, for example, the vpc that was just created or the database credentials in order to pass them to the step? I don't know their IDs, as auto-generating them is the best practice.

In this example code I have included some ??? comments on lines where I'm trying to access stack information from outside of the stack that clearly will not work.

example code:

class DemoStack(Stack):
    def __init__(...):
        ...
        vpc = ec2.Vpc(...)
        # RDS
        rds_instance_props = rds.InstanceProps(
            ...
            vpc=vpc,
        )
        rds_cluster = rds.DatabaseCluster(
            self, "DemoDatabase",
            credentials=rds.Credentials.from_generated_secret(
                "DemoDBCreds", secret_name="demo-db-login"
            ),
            instance_props=rds_instance_props,
        )
        ...

class DemoApp(Stage):
    def __init__(...):
        ...
        DemoStack(self, "DemoStack")

class DemoPipelineStack(Stack):
    def __init__(...):
        ...
        pipeline = pipelines.CodePipeline(...)
        pipeline.add_stage(
            DemoApp(...),
            post=[
                pipelines.CodeBuildStep(
                    ...,
                    vpc=vpc, # ???
                    security_groups=rds_cluster.security_groups, # ???
                    env={
                        "DB_HOST": rds_cluster.credentials.hostname, # ???
                        "DB_PASS": rds_cluster.credentials.password, # ???
                        ...,
                    }
                )
            ]
        )

I'm very open to other methods of achieving the same result, and solutions in non-Python languages. I have a docker image that successfully performs the necessary operations if I execute it by hand. I just need to make it run automatically, exactly once during each deployment after the stack with the RDS instance is good to go. It just needs that information so that it has access to the RDS instance.

Answer 1

This is only partially possible.

The goal of CDK pipelines is to create a CodePipeline which deploys your CDK code. This means, deployment happens in this order:

CDK creates Pipeline Stack -> Pipeline stack creates DemoStack

The VPC and security group configuration of the CodeBuild project defined in Pipeline stack are required when you create the stack. Since, the Pipeline is supposed to be the one creating the DemoStack containing these resources, this isn't possible. In theory, you might be able to create DemoStack manually without the Pipeline, export the VPC and security group ids as stack outputs using CfnOutput and then import the values using Fn.ImportValue in your PipelineStack. However, this would complicate your initial pipeline deployment setup and make it non-standard.

---

For values you need as environment variables in CodeBuild, you can use env_from_cfn_outputs. I haven't tested the below code, only creating it based on documentation references, but you can use it as a starting point.

class DemoStack(Stack):
    def __init__(...):
        ...
        vpc = ec2.Vpc(...)
        # RDS
        rds_instance_props = rds.InstanceProps(
            ...
            vpc=vpc,
        )
        self.rds_cluster = rds.DatabaseCluster(
            self, "DemoDatabase",
            credentials=rds.Credentials.from_generated_secret(
                "DemoDBCreds", secret_name="demo-db-login"
            ),
            instance_props=rds_instance_props,
        )
        ...

class DemoApp(Stage):
    def __init__(...):
        ...
        demo_stack = DemoStack(self, "DemoStack")
        self.hostname = CfnOutput(self, "hostname", value=demo_stack.rds_cluster.cluster_endpoint.hostname)
        self.credentials_secret_name = CfnOutput(self, "credentials_secret", value=demo_stack.rds_cluster.secret.secret_name)

class DemoPipelineStack(Stack):
    def __init__(...):
        ...
        pipeline = pipelines.CodePipeline(...)
        demo_app = DemoApp(...)
        pipeline.add_stage(
            demo_app,
            post=[
                pipelines.CodeBuildStep(
                    ...,
                    env_from_cfn_outputs={
                        "DB_HOST": demo_app.hostname,
                        "DB_PASS_SECRET": demo_app.credentials_secret_name,
                        # Make a call to secretsmanager get-secret-value to get the password value.
                        # DO NOT configure the password as a direct stack output.
                        ...,
                    }
                )
            ]
        )