0

I need to fetch data from an https url. I've learned that aiohttp doesn't support https proxy. Is there a way to overcome this. I've read lots of previous replies but couldn't find anything.

async def fetch(url, session):
    async with session.get(url, proxy=proxies['https']) as response:
        res_dic = await response.json()
        status = response.status

loop = asyncio.get_event_loop()
with ClientSession(headers=headers) as session:
    loop.run_until_complete(*[asyncio.ensure_future(fetch(url, session)) for url in urls])
loop.close()

I have my proxies like this

proxies = {
    "https": "https://{}:{}@{}".format(username, password, proxy_url),
    "http": "http://{}:{}@{}".format(username, password, proxy_url)
}
aksh02
  • 178
  • 1
  • 8
  • This may or may not help: httpx supports https proxies. See https://www.python-httpx.org/advanced/#http-proxying – dirn Sep 09 '20 at 11:56
  • `with httpx.Client(headers=headers, proxies=proxies) as client:` `resp = client.get(url)` `print(resp.json())` @dim I tried this and i get the error `httpx.ConnectError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1108)` – aksh02 Sep 09 '20 at 12:52

0 Answers0