0

I have created up an EMR cluster with Amazon S3 SSE-S3 Enabled.

Also I configured

fs.s3.enableServerSideEncryption = true
fs.s3.serverSideEncryption.kms.keyId = key-id

in emrfs-site.xml

I am writing my data as follows to s3 

aws s3 cp /home/hadoop/test.csv s3://my-bucket/enc/ --sse aws:kms /key-id-     here-/

This Will save my data encrypted with kms, But I dont want to send any parameters for encryption and whatever data sent from current EMR cluster should be encrypted.Just by 

aws s3 cp /home/hadoop/test.csv s3://my-bucket/enc/

Any way how I can do it ?

Eric
  • 2,636
  • 21
  • 25
Sanket_patil
  • 301
  • 1
  • 10

2 Answers2

1

No, it doesn't work that way.

If you want your target object encrypted, you will need to provide appropriate encryption related request headers

http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html

S3 decrypts copied objects for copying, then re-encrypts them at the destination, and this appears to be true even if the keys are the same.

Michael - sqlbot
  • 169,571
  • 25
  • 353
  • 427
0

Try enabling default bucket encryption and specified the key you want to use to encrypt all objects in the bucket. Post that all data written should be encrypted by default with your keys. I tested writing a file using boto in an encrypted bucket and that object does get encrypted with bucket key. I am not sure what is different with EMR

https://docs.aws.amazon.com/AmazonS3/latest/userguide/default-bucket-encryption.html

Ajak6
  • 727
  • 5
  • 17