2

What I'm trying to do is to connect a s3 bucket from my EC2 machine.

This error comes up if I don't set the endpoint_url in s3fs.S3FileSystem().

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/s3fs/core.py", line 246, in _call_s3
    out = await method(**additional_kwargs)
  File "/usr/local/lib/python3.7/site-packages/aiobotocore/client.py", line 155, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/s3fs/core.py", line 1064, in _info
    out = await self._simple_info(path)
  File "/usr/local/lib/python3.7/site-packages/s3fs/core.py", line 984, in _simple_info
    **self.req_kw,
  File "/usr/local/lib/python3.7/site-packages/s3fs/core.py", line 265, in _call_s3
    raise translate_boto_error(err)
PermissionError: Access Denied

However, I was able to fix it adding the endpoint_url in s3fs.S3FileSystem() to specify the region.

s3 = s3fs.S3FileSystem(key=MYKEY,
                       secret=SECRET,
                       client_kwargs={
                           'endpoint_url':'https://s3.ap-northeast-2.amazonaws.com'
                           })

I'm wondering if s3fs requires the specific region because there is no strict guidance in the documentation..!

Thanks for your help in advance.

Andrew Gaul
  • 2,296
  • 1
  • 12
  • 19
Trey Yi
  • 69
  • 6

1 Answers1

0

it seems that s3fs surely requires the specific region. if you've specified profile in your aws credentials and specified region in your aws config, you can use profile like this:

from aiobotocore.session import AioSession
s3=s3fs.S3FileSystem(session=AioSession(profile='your_profile_name'))

It works for me. Tips: 'your_profile_name' means profile in aws credentials, like this:

[your_profile_name]
aws_access_key_id = AKIAxxxxxxx
aws_secret_access_key = xxxxxxxxx
Anthony
  • 111
  • 1
  • 3