Ansible S3 object upload to Backblaze returns 403 for PutObjectAcl

Question

I am trying to use Ansible's community S3 task to upload a file to S3 compatible Backblaze but alas no success.

My task definition is:

- name: Install required dependencies
  pip: name=boto3

- name: Copy backup files to S3 bucket
  amazon.aws.aws_s3:
    bucket: personal-backup
    object: "{{ inventory_hostname }}/{{ item }}"
    src: "{{ item }}"
    mode: put
    permission: "private"
    s3_url: "https://s3.us-west-002.backblazeb2.com"
  environment:
    AWS_ACCESS_KEY: "<redacted>"
    AWS_SECRET_ACCESS_KEY: "{{ personal_backup_secret }}"
  loop: "{{ backup_files }}"

Which seems like it should be correct. But I keep getting this error:

{"ansible_loop_var": "item", "boto3_version": "1.20.37", "botocore_version": "1.23.37", "changed": false, "error": {"code": "AccessDenied", "message": "not entitled"}, "item": "/home/user/file.txt", "msg": "Unable to set object ACL: An error occurred (AccessDenied) when calling the PutObjectAcl operation: not entitled", "response_metadata": {"host_id": "aZWQ4m", "http_headers": {"cache-control": "max-age=0, no-cache, no-store", "content-length": "139", "content-type": "application/xml", "date": "Sun, 16 Jan 2022 23:40:01 GMT", "x-amz-id-2": "aZWQ4mDC", "x-amz-request-id": "9c8d9f2f6"}, "http_status_code": 403, "request_id": "9c8d9f2f61", "retry_attempts": 0}}

Which is odd because according to BackBlaze documentation:

The Put Object ACL call only supports the same canned ACL values mentioned previously. The call will succeed only when specified ACL matches the ACL of the bucket.

Meaning that the object permission ("private") needs to match the bucket permission... but my bucket is marked as private so that should already be okay. The application key I am using does have "writeFiles" permission (confirmed in UI) which should have access to "Put Object ACL" as per the same Backblaze documentation page.

So, I am surprised it doesnt work, and not sure how to work around it. Anyone get Ansible working with Backblaze before? Thanks!

Edit: Oh, the upload actually worked (it takes a while before it shows up in the UI). But it does still error out which is not good for an Ansible task. I guess it must upload as one step internally, then try to adjust ACL as another step which is the part that fails.

I don't know if you just typoed it here, but the env-var is actually named `AWS_ACCESS_KEY_ID` not just `AWS_ACCESS_KEY`; it's also mysterious why you would use `environment:` for that when the task has dedicated properties for those credentials — mdaniel, Jan 17 '22 at 18:50
Typically, you put secrets in ENV variables so they never ever show up in output console or logs, even if you do high verbosity. — NewUser, Jan 19 '22 at 04:22
No need to spread rumors when trivially testing that with `-vvvv` shows `"secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",` as well as having [the source code](https://github.com/ansible-collections/amazon.aws/blob/3.0.0/plugins/module_utils/ec2.py#L195-L196) to see they are marked as exempt from logging — mdaniel, Jan 19 '22 at 16:39
@mdaniel It is best practice in many environments to store sensitive values in environment variables rather than config files, just so there's no chance of them being checked into a repository. — metadaddy, Jan 27 '22 at 18:49

score 2 · Accepted Answer · edited Oct 14 '22 at 23:02

2

For those who stumble upon this ... the solution is to generate an access key that has "writeBuckets" permission (this is set by selecting "all" in the screenshot below, instead of selecting a particular bucket).

This is obviously a non-ideal solution because then you have to have that application key get access to all buckets, not just the one that you want for that particular application use case (which is against security best practices). That doesn't seem like it should be necessary for all because you are only ever interacting with that single bucket.

I am going to give feedback to Backblaze team that it is not ideal.

edited Oct 14 '22 at 23:02

metadaddy

4,234
1
22
46

answered Jan 20 '22 at 20:04

NewUser

73
5

1

This is definitely a bug - Put Object ACL should work for bucket-scoped app keys. I've created a ticket to fix this. – metadaddy Jan 27 '22 at 21:00
1

It took a while, but we just deployed the fix. Bucket-scoped app keys no longer need `writeBuckets` permission. – metadaddy Oct 07 '22 at 00:06
1

@metadaddy Awesome! Thanks for making the improvements! And thanks for replying here letting me and community know! Now I can regenerate my keys with more appropriate permissions ❤️ – NewUser Oct 08 '22 at 01:58

Ansible S3 object upload to Backblaze returns 403 for PutObjectAcl

1 Answers1