As the title mentioned..
A little more detailed description of my usage:
I use Caddy as reverse proxy for aiohttp. And then Cloudflare in-front of Caddy. This is unrelated to Caddy, because using gray-clouded DNS entry (no proxy from Cloudfare) makes it work fine.
But whenever the traffic goes through Cloudflare, the websocket connection just dies after 2 minutes with 1006 closure code. No other message attached to the close frame.
I could reproduce it with the example websocket code snippet from the aiohttp documentation. (https://docs.aiohttp.org/en/stable/web_quickstart.html#websockets)