1

I can't set proxy server for S3FS Python API. As S3FS's Config class is imported from botocore there is no S3FS documentation about it. So, I have read this question and also botocore documentation.

However, I couldn't manage to get botocore Config class working with S3FS.

I have tried:

from s3fs.core import Config
proxy = Config(proxies={'https':<my_proxy>})

fs = s3fs.S3FileSystem(key=MY_S3_KEY, secret=MY_S3_SECRET, config_kwargs=proxy)
fs.put(FILE_NAME, PATH)

I have also tried:

from s3fs.core import Config

fs = s3fs.S3FileSystem(key=MY_S3_KEY, secret=MY_S3_SECRET, config_kwargs=Config(proxies={'https':<my_proxy>}))
fs.put(FILE_NAME, PATH)

I have also tried to set same proxy for http. The proxy works fine from this computer, I have sucessfully tested several times from command line.

I always get Connection Timeout error message:

ConnectTimeoutError: Connect timeout on endpoint URL:....

Any suggestion is appreciated. Thanks

Andrew Gaul
  • 2,296
  • 1
  • 12
  • 19
MarlosB
  • 185
  • 2
  • 11

1 Answers1

2

Here are two ways to do it.

1 - Set an environment variable with the proxy:

os.environ['https_proxy'] = 'http://my_proxy_url'
s3 = s3fs.S3FileSystem(anon=False)

2 - pass in the proxy in the constructor:

s3 = s3fs.S3FileSystem(anon=False, config_kwargs={'proxies': {'https': 'http://my_proxy_url'}})
sandlb
  • 198
  • 2
  • 9