How do I use HTTPS proxy with aiohttp?

Question

How do I use HTTPS proxy with aiohttp in Python?

I have seen the docs where it is said I need to upgraded to HTTPS via the HTTP CONNECT method. But I have not found any complete Pythonic example how to do it? I have also viewed the questions Using https proxy with aiohttp and Does aiohttp support HTTPS proxies. They refer to the original documantation without details.

Let's suppose I have an HTTPS proxy with IP 1.2.3.4 and port 3128 and I want to request https://stackoverflow.com/ through it. How would the Python code (using asynchronous aiohttp) for it look like?

I tried this with the error:

import asyncio
import aiohttp

async def main():
    proxy = f"https://1.2.3.4:3128"
    async with aiohttp.ClientSession() as session:
        async with session.get("https://stackoverflow.com/", proxy=proxy) as response:
            print(response)

asyncio.run(main())

The error:

ValueError: Only http proxies are supported

If you must connect to the proxy via HTTPS, then it's not supported, as written in [the documentation](https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support). If your proxy allows connections via HTTP and then aiohttp can negotiate an upgrade of the connection, that'll work. Just pass `"http://..."` as the proxy address then. — deceze, Mar 09 '21 at 11:15
Thank you, it worked. There is no way to force use of HTTPS explicitly? — Fomalhaut, Mar 09 '21 at 12:29
Apparently not. Why is that important? If the target site is HTTPS, the connection (end to end, between Python and the target) will get upgraded to HTTPS anyway. — deceze, Mar 09 '21 at 12:37

score 1 · Answer 1 · answered Oct 31 '22 at 18:16

You need to turn off the ssl_context to http server, see this thread for more information https://github.com/aio-libs/aiohttp/issues/206

async with aiohttp.ClientSession(timeout=timeout) as client:
    async with client.get(
        proxy=url,
        url=server_url,
        data=b"Test content!",
        ssl_context=None,
    ) as resp:
        ...

How do I use HTTPS proxy with aiohttp?

1 Answers1