8

I have an S3 bucket with several folders and each folder contains several files. Because the number of files keep growing, I'm thinking to create a S3 lifecycle rule to delete files older than 14 days.

My concern is that this rule might also delete a folder if it's empty. Is this the case or will this rule only delete files/objects?

Thanks.

GreenTeaTech
  • 231
  • 1
  • 3
  • 7

2 Answers2

8

http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html

Amazon S3 has a flat structure with no hierarchy like you would see in a typical file system. However, for the sake of organizational simplicity, the Amazon S3 console supports the folder concept as a means of grouping objects. Amazon S3 does this by using key name prefixes for objects.

In other words, folders don't actually exist on S3.

ceejayoz
  • 32,910
  • 7
  • 82
  • 106
  • 2
    I know that, but for the sake of simplicity I used the word 'folder'. The question is if an object ("folder") is older than the lifecycle rule but it has other objects within it, would this object "folder" be deleted? If so, that would also delete other objects that are not older than the lifecycle but they are grouped objects within that "folder". It's hard for me to explain this without using the word "folder". – GreenTeaTech Sep 30 '16 at 21:20
  • 2
    My point is that the folder doesn't actually exist. It's a fake concept for the console only. Nothing in how S3 actually works involves folders - it's just an object store. `foo/bar/something.jpg` and `foo/bar/something-else.jpg` have absolutely no linkage to each other on S3. `foo/` and `foo/bar/` don't exist at all. **An "empty" folder doesn't exist.** A folder in the console just means the console saw some `/` characters in your object names. – ceejayoz Sep 30 '16 at 21:30
  • 4
    Technically, something that *looks like* an "empty folder" can exist (but with no real meaning)... if you manually create a "folder" called `foo` in the console, what you've actually done is created a zero-byte placeholder object with the key `foo/` which the console will use to display an (initially) "empty folder." If a lifecycle rule causes this to be removed, it doesn't matter, because the folder illusion is still present as long as there are any objects called `foo/*`. If all such objects are later removed, the "folder" disappears, but reappears if files named `foo/*` are again created. – Michael - sqlbot Oct 01 '16 at 18:04
0

I've tested this concept as we use the s3 repository for SFTP storage. I applied a 7-day policy to expire the current version of objects and permanently delete the previous version of objects" enter image description here

The subfolders DO get deleted when the retention policy set to: "This rule applies to all objects in the bucket". Hopefully, an option will be added soon to AWS retention policy so that it can be set to no include Folders.

enter image description here

Hope this helps !

BrettKennard
  • 41
  • 4