How to reverse tunnel over HTTP only firewall

Question

I have a server ("A") running behind a firewall, which only serves HTTP (I kid you not). I have no control over the firewall, but "A" can connect to server "B", though the reverse connection from B to A is not possible due to the firewall. I've successfully installed Jenkins nodes on "A" since they have an option to use web sockets to connect back to the master node on "B". As a result I've been trying to set up something similar in order to be able to access resources on "A". I thought of using websocket reverse proxying but the firewall does not allow ws connections. I am missing a piece of the puzzle or going in the wrong direction. Any helps highly appreciated