2

I'm using a container that simulate a S3 server running on http://127.0.0.1:4569 (with no authorization or credentials needed)

and I'm trying to simply connect and print a list of all the bucket names using python and boto3

here's my docker-compose:

s3:
image: andrewgaul/s3proxy
environment:
  S3PROXY_AUTHORIZATION: none
hostname: s3
ports:
  - 4569:80
volumes:
  - ./data/s3:/data

here's my code:

s3 = boto3.resource('s3', endpoint_url='http://127.0.0.1:4569')

for bucket in s3.buckets.all():
    print(bucket.name)enter code here

here's the error message that I received:

botocore.exceptions.NoCredentialsError: Unable to locate credentials

I tried this solution => How do you use an HTTP/HTTPS proxy with boto3?

but still not working, I don't understand what I'm doing wrong

Andrew Gaul
  • 2,296
  • 1
  • 12
  • 19
Louis W.
  • 81
  • 1
  • 3
  • 9
  • Does this answer your question? [How do you use an HTTP/HTTPS proxy with boto3?](https://stackoverflow.com/questions/33480108/how-do-you-use-an-http-https-proxy-with-boto3) – Alex R Mar 19 '20 at 00:36

1 Answers1

1

First, boto3 always try to handshake with S3 server with AWS API key. Even your simulation server don't need password, you still need to specify them either in your .aws/credentials or inside your program. e.g. 

[default] 
aws_access_key_id = x
aws_secret_access_key = x

hardcoded dummy access key example

import boto3
session = boto3.session(
  aws_access_key_id = 'x', 
  aws_secret_access_key = 'x')

s3 = session.resource('s3', endpoint_url='http://127.0.0.1:4569')

Second, I don't know how reliable and what kind of protocol is implemented by your "s3 simulation container". To make life easier, I always suggest anyone that wants to simulate S3 load test or whatever to use fake-s3

ofrommel
  • 2,129
  • 14
  • 19
mootmoot
  • 12,845
  • 5
  • 47
  • 44
  • Thank you :) I tried it => session = boto3.session.Session(aws_access_key_id = ' ', aws_secret_access_key = ' ') => s3 = session.resource('s3', endpoint_url='http://127.0.0.1:4569') => print(s3.get_available_subresources()) and it worked just fine i get what i asked for ( an array of all the subresources available ) **BUT** when i tried to use one of that subresources to upload a file => s3.Object(BUCKET_NAME, FILE_NAME).put(Body=open(FILE_NAME, 'rb')) i get this error message => ('Connection aborted.', ConnectionRefusedError(111, 'Connection refused')) – Louis W. Jan 12 '18 at 10:24
  • @LouisW please as a new question, and remember to mentioned your "localised S3 instance" – mootmoot Jan 12 '18 at 15:43
  • it depends maybe on version of boto but for me the following code was returning an error, using session = boto3.Session( <--- Session (with Capital S) instead of session work better. – Gabriel Feb 28 '20 at 01:15