1

I have the following proxy list:

proxies = [
    {
        'http': 'http://user:pass@10.10.1.10:3128',
        'https': 'http://user:pass@10.10.1.10:3128'
    },
    {
        'http': 'http://user:pass@10.10.1.10:3128',
        'https': 'http://user:pass@10.10.1.10:3128'
    },
    {
        'http': 'http://user:pass@10.10.1.10:3128',
        'https': 'http://user:pass@10.10.1.10:3128'
    }
]

Here I have my session:

session = requests.Session()
session.proxies=proxies
session.get(url)

In the Requests documentation I only see this:

proxies = {
    'http': 'http://user:pass@10.10.1.10:3128',
    'https': 'http://user:pass@10.10.1.10:3128'
}

But I want to use multiple proxies, every time I make a request. Or everytime I use my session method to make a request.

def requests_retry_session(
    retries=3,
    backoff_factor=0.3,
    status_forcelist=(500, 502, 504),
    session=None,
):
    session = requests.Session()
    session.proxies=proxies
    retry = Retry(
        total=retries,
        backoff_factor=backoff_factor,
        status_forcelist=status_forcelist,
    )
    adapter = HTTPAdapter(max_retries=retry)
    session.mount('https://', adapter)
    return session

s = requests.Session()
s.auth = ('user', 'pass')
s.headers.update({'x-test': 'true'})

response = requests_retry_session(session=s).get(url)
Lukas
  • 53
  • 6
  • https://stackoverflow.com/a/58500280 – Seb Nov 29 '19 at 01:20
  • You can't assign all proxies - you have to change proxies on your own. – furas Nov 29 '19 at 01:21
  • I thought about using `random` as follows: `session.proxies = random.choice(proxies)`, could you suggest a better approach? – Lukas Nov 29 '19 at 01:33
  • @Seb That would be useful if you want to test the proxies, my question is different. – Lukas Nov 29 '19 at 03:14
  • As I interpreted your question, you want to use a different proxy for every subsequent request. Is that not what you’re trying to do? – Seb Nov 29 '19 at 10:00
  • @Seb Yes, but the solution you sent me is pretty much a for loop making a request for each proxy to the same url, basically it testing the proxies. – Lukas Nov 30 '19 at 01:38
  • I would have thought it would be obvious to only take the lines you need, and adapt the rest to your use case. – Seb Nov 30 '19 at 01:40

0 Answers0