How to delete files in S3 using CodePipeline

Question

I have created a CodePipeline that takes input(files) from CodeCommit repository and deploys files to S3 bucket. When I delete files in CodeCommit repository, pipeline doesn't delete the file in the corresponding S3 bucket. In short, I want my S3 bucket to be in sync with CodeCommit repository. How can I achieve this?

My Pipeline doesn't have CodeBuild stage.

AWS-CodeCommit --> CodePipeline --> Deploy-to-Cross-Account-S3

score 3 · Answer 1 · answered Sep 24 '19 at 12:14

3

Unfortunately, currently there is no predefined configuration of an S3 Deploy action to delete an S3 object from the pipeline itself.

Docs

The workaround for now will be to add another action on the pipeline to use CodeBuild to perform the objects cleanup before/after deploying them.

answered Sep 24 '19 at 12:14

shariqmaws

8,152
1
16
35

1

So I guess this means adding a Build stage that just deletes all the files in the s3 bucket deploy location. I noticed that CodePipeline replaces all the existing objects, the deleted objects (which are still in the bucket) have an older timestamp. So the Build stage can be `aws s3 rm s3://my-bucket/deploy-path/deployment --recursive`, and then the deployment will write all the current objects there. – Merlin Dec 02 '21 at 06:22

score 1 · Answer 2 · answered Feb 08 '22 at 02:12

1

You can add a CodeBuild stage of:

      - aws s3 sync . s3://[bucket_name] --delete

Not sure how to make it work cross-account, tho.

answered Feb 08 '22 at 02:12

Lottike

63
5

score 0 · Answer 3 · answered Mar 12 '23 at 17:58

I created a custom AWS construct to replace S3DeployAction to deploy to S3 and delete old files. The construct diffs the changes using a lambda function to delete old files and upload new files to S3.

To use it just replace S3DeployAction with S3SyncDeployAction.

import { S3SyncDeployAction } from "@monim67/s3-sync-deploy-action";

const deployAction = new S3SyncDeployAction({
  actionName: 'S3Deploy',
  bucket: targetBucket,
  input: sourceOutput,
});

See installation and usage details of the Typescript construct here.

How to delete files in S3 using CodePipeline

3 Answers3

Linked