S3 Sync Not Writable Warning

Question

I am trying to use s3cmd sync (vanilla command remote to local), but for each file I get a not writable: Operation not permitted warning (see below). The files appear to download correctly, however, and if I do an ls on the file it is there. I tried using the --disable-multipart flag but this made no difference. I am using s3cmd version 1.5.0-rc1.

Is there any reason why I might be receiving these warnings?

WARNING: MD5 signatures do not match: computed=11ff15bf.., received="b119cb85b109c6.."
WARNING: ./path/part-r-00003.gz.parquet not writable: Operation not permitted
s3://bucket/path/part-r-00003.gz.parquet -> <fdopen>  [31 of 72] 36101458 of 36101458   100% in 2s    16.30 MB/s  done

score 31 · Answer 1 · answered Dec 06 '16 at 15:54

31

A likely reason is that the S3 objects have been stored with filesystem attributes. When you sync them to a local directory, by default s3cmd sync will try to restore these attributes, including uid,gid ownership.

To prevent s3cmd from doing this, run it with the --no-preserve option, e.g.:

s3cmd sync --no-preserve s3://bucket/my/key/prefix .

answered Dec 06 '16 at 15:54

Stragulus

1,023
11
10

1

This worked for me; believe it should have been marked as the answer. – GDB Jun 19 '17 at 13:35
1

This is the correct answer. Should be marked as such. – nrm Nov 30 '17 at 23:53

S3 Sync Not Writable Warning

1 Answers1