0

I'm using a good example to delete all file versions, delete markers and objects inside an Amazon S3 bucket and folder https://stackoverflow.com/a/53872419/387774

import boto3
bucket = "my-bucket-name"
filename = "my-folder/filename.xml"
client = boto3.client("s3")
paginator = client.get_paginator("list_object_versions")
response_iterator = paginator.paginate(Bucket=bucket)
for response in response_iterator:
    versions = response.get("Versions", [])
    versions.extend(response.get("DeleteMarkers", []))
    for version_id in [
        x["VersionId"]
        for x in versions
        if x["Key"] == filename and x["VersionId"] != "null"
    ]:
        print("Deleting {} version {}".format(filename, version_id))
        client.delete_object(Bucket=bucket, Key=filename, VersionId=version_id)

The problem I'm facing is there are thousands of files and their versions. I do not know the names of all those files. I would like to delete all of them using some wildcard like filename = "my-folder/*.xml"

What changes can I make?

smac2020
  • 9,637
  • 4
  • 24
  • 38
Some Java Guy
  • 4,992
  • 19
  • 71
  • 108
  • Related: https://stackoverflow.com/questions/43326493/what-is-the-fastest-way-to-empty-s3-bucket-using-boto3 – jarmod Aug 19 '21 at 21:04
  • 1
    Are you saying that Versioning is enabled on the bucket, and some objects that you wish to delete have multiple versions -- all of which you wish to remove? How many objects are in the bucket? Is this a one-off activity, or a continuing activity? I'm asking to figure out whether obtaining an [Amazon S3 Inventory](https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-inventory.html) listing would be helpful. – John Rotenstein Aug 19 '21 at 22:02
  • @JohnRotenstein - Yes, there are multiple versions of the same file that I would like to remove. There are many objects possibly close to 50k files. This is one off activity. – Some Java Guy Aug 31 '21 at 03:10
  • Is your goal to completely empty the bucket or everything, or do you you simply wish to delete the non-current version of objects? – John Rotenstein Aug 31 '21 at 04:32
  • Please do not post [duplicate questions](https://stackoverflow.com/questions/68986110/python-script-to-delete-all-versions-including-delete-marker-in-s3-for-files-wit). – John Rotenstein Aug 31 '21 at 06:37
  • Unfortunately boto3 is not able to filter via suffix. I think you can use prefix to get all the filenames in that dir via list_objects_v2. And then keep the names of the files that ends with your extension. Then you can delete those objects. – michaelgbj Apr 14 '22 at 18:56

0 Answers0