2

I am testing out MLOps using SageMaker studio and am creating a project using a template for MLOps provided by SageMaker: MLOps template for model building, training, and deployment with third-party Git repositories using CodePipeline

I am getting this error when creating the project

Your project couldn't be created
Studio encountered an error when creating your project. Try recreating the project again.

Client error: Provisioning failed with error: Errors from CloudFormation: [{LogicalResourceId : SC-493356053890-pp-khphda3aqa5qq, ResourceType : AWS::CloudFormation::Stack,

StatusReason : The following resource(s) failed to create: [SageMakerModelDeploySeedCodeCheckinProjectTriggerLambdaInvoker, SageMakerModelBuildSeedCodeCheckinProjectTriggerLambdaInvoker]. Rollback requested by user.},

{LogicalResourceId : SageMakerModelBuildSeedCodeCheckinProjectTriggerLambdaInvoker, ResourceType : AWS::CloudFormation::CustomResource, StatusReason : Resource creation cancelled}, {LogicalResourceId : SageMakerModelDeploySeedCodeCheckinProjectTriggerLambdaInvoker, ResourceType : AWS::CloudFormation::CustomResource,

StatusReason : Received response status [FAILED] from custom resource. Message returned: Codebuild to checkin seedcode has status FAILED (RequestId: ab735779-d179-4714-af64-f0e17b5b671b)},

{LogicalResourceId : SageMakerModelDeploySeedCodeCheckinProjectTriggerLambdaInvoker, ResourceType : AWS::CloudFormation::CustomResource, StatusReason : Resource creation Initiated}, {LogicalResourceId : SageMakerModelPipelineBuildProject, ResourceType : AWS::CodeBuild::Project, StatusReason : Resource creation Initiated}, {LogicalResourceId : ModelDeployBuildProject, ResourceType : AWS::CodeBuild::Project, StatusReason : Resource creation Initiated}, {LogicalResourceId : GitSeedCodeCheckinProjectTriggerLambda, ResourceType : AWS::Lambda::Function, StatusReason : Resource creation Initiated}, {LogicalResourceId : GitSeedCodeCheckinProject, ResourceType : AWS::CodeBuild::Project, StatusReason : Resource creation Initiated}, {LogicalResourceId : ModelBuildSagemakerCodeRepository, ResourceType : AWS::SageMaker::CodeRepository, StatusReason : Resource creation Initiated}, {LogicalResourceId : ModelDeploySagemakerCodeRepository, ResourceType : AWS::SageMaker::CodeRepository, StatusReason : Resource creation Initiated}, {LogicalResourceId : MlOpsArtifactsBucket, ResourceType : AWS::S3::Bucket, StatusReason : Resource creation Initiated}, {LogicalResourceId : ModelDeployTestProject, ResourceType : AWS::CodeBuild::Project, StatusReason : Resource creation Initiated}, {LogicalResourceId : WaitHandle, ResourceType : AWS::CloudFormation::WaitConditionHandle, StatusReason : Resource creation Initiated}, {LogicalResourceId : SC-493356053890-pp-khphda3aqa5qq, ResourceType : AWS::CloudFormation::Stack, StatusReason : User Initiated}].

The IAM role attached to the SageMaker studio has the following policies attached

{
    "AttachedPolicies": [
        {
            "PolicyName": "AWSCodePipelineFullAccess",
            "PolicyArn": "arn:aws:iam::aws:policy/AWSCodePipelineFullAccess"
        },
        {
            "PolicyName": "AmazonSageMakerFullAccess",
            "PolicyArn": "arn:aws:iam::aws:policy/AmazonSageMakerFullAccess"
        },
        {
            "PolicyName": "AmazonSageMaker-ExecutionPolicy-20211020T164534",
            "PolicyArn": "arn:aws:iam::493356053890:policy/service-role/AmazonSageMaker-ExecutionPolicy-20211020T164534"
        },
        {
            "PolicyName": "ECR_FullAccess",
            "PolicyArn": "arn:aws:iam::493356053890:policy/ECR_FullAccess"
        },
        {
            "PolicyName": "AmazonSageMakerServiceCatalogProductsUseRole-20211029T121670",
            "PolicyArn": "arn:aws:iam::493356053890:policy/service-role/AmazonSageMakerServiceCatalogProductsUseRole-20211029T121670"
        },
        {
            "PolicyName": "SagemakerDomainKeyAccess-asp",
            "PolicyArn": "arn:aws:iam::493356053890:policy/SagemakerDomainKeyAccess-asp"
        },
        {
            "PolicyName": "AWSLambda_FullAccess",
            "PolicyArn": "arn:aws:iam::aws:policy/AWSLambda_FullAccess"
        }
    ]
}

When creating a new project I pass,

  • The URLs to the training and deployment git repositories
  • Respective username/repositoryname
  • ARN of the CodeStar connection to GitHub with a tag sagemaker=true (access to all the repos in my account)

What am I missing here?

Abhishek Purandare
  • 53
  • 1
  • 5
  • From your error message it looks like what has failed is pushing the seed code to your GitHub repositories. This template uses CodeBuild to push code to your GitHub repos and CodeBuild is triggered by a Lambda function. It is likely that this is an issue with the connection between CodeBuild and GitHub. – Kirit Thadaka May 19 '22 at 22:42
  • Okay. I have added permission for CodeStarConnection to the sagemaker role and it is now working correctly. Thank you. – Abhishek Purandare May 25 '22 at 06:00
  • I've added ```CodeStarFullAccess``` to the execution role and still get the error – momo668 Apr 14 '23 at 03:55

0 Answers0