0

I'd like to implement an automatic code pipeline with k8s.

To do that, I selected CodePipeline and AWS CDK.

I initialized my cdk project and wrote down these codes.


// path: /bin/my-project-k8s-cdk.ts
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import {DeploymentStack} from "../lib/kubeflow-helm-deployment-stack";

const app = new cdk.App();
new DeploymentStack(app, "DeploymentStack", {
  env: { account: '~~~~~~~~', region: 'ap-northeast-2' },
});

app.synth();

// path: /lib/kubeflow-helm-deployment-stack.ts
export class DeploymentStack extends Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const pipeline = new CodePipeline(this, "EksDeploymentPipeline", {
      pipelineName: "EksDeploymentPipeline",
      synth: new CodeBuildStep("Synth", {
        input: CodePipelineSource.connection(GithubConfig.GITHUB_REPO, "main", {
          connectionArn: GithubConfig.GITHUB_CODESTAR_CONNECTION_ARN
        }),
        commands: [
          "npm install",
          "npm run build",
          "npx cdk synth"
        ],
        rolePolicyStatements: [
          new PolicyStatement({
            actions: ['sts:AssumeRole'],
            resources: ['*'],
            conditions: {
              StringEquals: {
                'iam:ResourceTag/aws-cdk:bootstrap-role': 'lookup',
              },
            },
          }),
        ],
      }),
    });

    const eksDeploymentStage = new EksDeploymentStage(this, "EksDeploymentStage", {
      env: { account: this.account, region: this.region }
    });

    pipeline.addStage(eksDeploymentStage);
  }
}

// path: /lib/kubeflow-helm-eks-deployment-stage.ts
export class EksDeploymentStage extends Stage {

  constructor(scope: Construct, id: string, props?: StageProps) {
    super(scope, id, props);

    new MyEksStack(this, 'MyEksStack')
  }
}

// path: /lib/my-eks-stack.ts
export class MyEksStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const accountId = this.account;
    const clusterName = 'MyKubeFlowCluster';
    const kubernetesVersion = KubernetesVersion.V1_21;
    const albControllerVersion = AlbControllerVersion.V2_4_1;

    const kubernetesApiAccessPolicy = new PolicyStatement({
      actions: [
        'eks:DescribeCluster',
        'eks:AccessKubernetesApi',
        'cloudformation:DescribeStacks',
      ],
      resources: [
        `arn:aws:eks:*:${accountId}:cluster/*`,
      ]
    });

    const eksClusterMasterRole = new Role(this, "EksClusterMasterRole", {
      assumedBy: new AccountPrincipal(accountId),
      roleName: "EksClusterMasterRole",
      inlinePolicies: {
        "KubernetesApiAccess": new PolicyDocument({
          statements: [kubernetesApiAccessPolicy]
        })
      }
    });

    const kubeflowVpc = new Vpc(this, "KubeflowVpc", {
      vpcName: "KubeflowVpc",
      ipAddresses: IpAddresses.cidr("10.1.0.0/16"),
      maxAzs: 2,
      natGateways: 1,
      subnetConfiguration: [
        {
          name: "PrivateSubnet",
          subnetType: SubnetType.PRIVATE_WITH_EGRESS,
        },
        {
          name: "PublicSubnet",
          subnetType: SubnetType.PUBLIC
        }
      ]
    });

    const cluster = new Cluster(this, "KubeflowCluster", {
      clusterName: clusterName,
      version: kubernetesVersion,
      vpc: kubeflowVpc,
      defaultCapacity: 1,
      defaultCapacityInstance: new InstanceType("t2.small"),
      mastersRole: eksClusterMasterRole,
      albController: {
        version: albControllerVersion
      },
      outputClusterName: true
    });

    kubeflowVpc.publicSubnets.forEach(subnet => {
      Tags.of(subnet).add(`kubernetes.io/cluster/${cluster.clusterName}`, "owned")
    });

    kubeflowVpc.privateSubnets.forEach(subnet => {
      Tags.of(subnet).add(`kubernetes.io/cluster/${cluster.clusterName}`, "owned")
    });
  }
}

With those codes, the CodePipeline was generated correctly.

Above CodePipeline's structure is as fellows.

Source -> Build -> UpdatePipeLine -> Assets -> EksDeploymentStage

Before EksDeploymentStage everything was perfectly done.

However, when the pipeline arrived to EksDeploymentStage step, it was failed with this error message.

Circular dependency between resources: [My Resources list~~~~~~~~~~~~] (Service: AmazonCloudFormation; Status Code: 400; Error Code: ValidationError; Request ID: ~~~~~~~~~~-3ffc3b280a8a; Proxy: null)

What is the problem of my code...?

Jun
  • 451
  • 4
  • 16
  • https://stackoverflow.com/a/60418885/13126651 – Jatin Mehrotra Feb 15 '23 at 06:05
  • As the link above points out. Split shard resources (VPC, SecGroup, Role) to another stack. Move the cluster to a new stack and import it or pass in as part of stack setup. use custom resources. check the resources the tier 3 is setting up. you're already using policies with "*" from what I can tell. – lloyd Feb 18 '23 at 07:49
  • Your VPC construct depends on the cluster (it needs the cluster name to set the tags), while your cluster depends on the VPC. – gshpychka Feb 28 '23 at 11:20

0 Answers0