1
aws s3 sync /WordProcessing/DOCUMENTS s3://mybigbucket --delete --include "*" --exclude ".DS_Store"

If somehow .DS_Store made its way to the AWS S3 mybigbucket (from a previous backup), using the above command does not delete the .DS_Store files from the destination, even if I manually:

rm /WordProcessing/DOCUMENTS/.DS_Store

on the file server.

Am I missing the logic of how aws s3 sync works? It seems to me it should delete the .DS_Store from the destination on the S3 bucket if it has been removed from the source on the file server.

If I take off the --exclude and run the command again, it does delete .DS_Store from the destination when it has been removed from the file server. That doesn't seem right.

Am I taking the wrong approach? What I want to do, is nightly sync the file server and have it not transfer to the S3 bucket the thousands of .DS_Store files.

Edward_178118
  • 955
  • 4
  • 15
  • 33

1 Answers1

5

It looks like you've changed your parameters after the first sync.

When you use the --exclude it excludes all files or objects from the command that matches the specified pattern. If it matches, it will ignore any actions for that pattern.

If the file (.DS_Store) is already located in the s3 bucket and the --exclude option matches it, it will ignore it. Whether it is changed or gone from the data source.

You can do:

  1. Use the --exclude on the first and all subsequent sync's (this keeps the file off s3 at the outset)
  2. Delete the file from the localhost first, sync, then put the exclude in (this will remove the file if it's in s3, then prevent it from being added)
  3. Put the --exclude in and manually delete the file from s3 (this will prevent the file from being re-added to s3)

Reference

AWS cli - sync

kenlukas
  • 3,101
  • 2
  • 16
  • 26