1

I have a powershell script, that downloads all files form an S3 bucket, and then removes the files from the bucket. All the files I'm removing are stored in a subfolder in the S3 bucket, and I just want to delete the files but maintain the subfolders.

I'm currently using the following command to delete the files in S3 once the file has been downloaded from S3.

 Remove-S3Object -BucketName $S3Bucket -Key $key -Force

My problem is that if it removes all the files in the subfolder, the subfolder is removed as well. Is there a way to remove the file, but keep the subfolder present using powerhsell. I believe I can do something like this,

aws s3 rm s3://<key_to_be_removed> --exclude "<subfolder_key>"

but not quite sure if that'll work.

I'm looking for the best way to accomplish this, and at the moment, my only option is to recreate the subfolder via the script if the subfolder not longer exists.

helpo11
  • 47
  • 5
  • No, there is no way. Reason: S3 does not have folder per-se, your objects have a path or key and if that path contains slashes then there are "folders" (in the UI), if no object with key `/sub/something.file` exists then the "folder" `sub` does not exist either. – luk2302 Jul 14 '21 at 16:44
  • 1
    You can add an empty file with a key that is the folder. I believe there is even a content type that indicates that it is a folder. That's what AWS does in the console if you use the "create folder". – Jason Wadsworth Jul 14 '21 at 17:08
  • How are "disappearing folders" causing an issue for you? Is there a particular need to keep empty folders present? – John Rotenstein Jul 14 '21 at 23:35

2 Answers2

0

As already commented, S3 does not really have folders the way file systems do. The folders as presented by most S3 browsers are just generated based on the paths of the files/objects. If you upload an object/file named folder/file, the browsers will present folder as folder with file as a file in the folder. But technically, all that there is is the file/object folder/file. The folder does not exist on its own.

You can explicitly create a folder by creating an empty empty-named object with "in the folder": folder/. If you do that, it will appear the the folder exists even if there are no files in it. But if you do not do that, the virtual folder disappears once you remove all objects in the folder.

Now the question is whether your command removes even the empty named object representing the folder or not. I cannot tell that.

Martin Prikryl
  • 188,800
  • 56
  • 490
  • 992
0

The only way to accomplish having an empty folder is to create a zero-length object which has the same name as the folder you want to keep. This is actually how the S3 console enables you to create an empty folder.

You can check this by running $ aws s3 ls s3://your-bucket/folderfoo/ and observing an output object having length of zero bytes.

See more on this topic here.

StefanN
  • 527
  • 1
  • 4
  • 12