1

I'm running this brief script locally as well on an AWS EC2 intance in an ECS cluster:

 import requests 
 proxies = {'http':'http://user:pw@host:port','https':'http://user:pw@host:port'}
 r = requests.get('http://quotes.toscrape.com/', proxies=proxies)
 print r.status_code

When I run the script locally, I get a 200 status code, indicating that I successfully am able to connect to access the website via the proxy.

When I run the script on the AWS instance, I get a 407 proxy authentication error.

This is a common error that others have experienced (e.g. see here). I'm looking for a solution that allows me to authenticate the proxies WITHOUT having to whitelist the instance.

The reason being is that every time I run a new instance, I'd have to whitelist that instance too. I would rather just pass in the credentials to requests and authenticate the proxy directly.

Is this possible?

chris
  • 1,869
  • 4
  • 29
  • 52

1 Answers1

0

I would suggest you launch instances in the private subnet, and whitelist your NAT(s) EIP in this case you will only have to whitelist 1-4 IP addresses depending if you are using single NAT or a NAT per AZ (which is recommended).

Hopefully, it does make sense, feel free to ask additional questions

b.b3rn4rd
  • 8,494
  • 2
  • 45
  • 57