I am attempting to run a transparent stunnel4 proxy on a server for a websocket service.
The WS server is based on the Ratchet framework and thus does not support WSS, hence the need for a proxy.
When transparent is set to none in the stunnel.conf the websocket traffic gets through without incident and everything works perfectly besides the fact that as far as the WS server is concerned, all traffic is coming from 127.0.0.1.
However once transparent is set to source, the client just gets
WebSocket connection to 'wss://<ADDR>:32770/' failed: Error during WebSocket handshake: net::ERR_CONNECTION_RESET
As far as I can tell, the issues begin after setsockopt IP_TRANSPARENT: Operation not permitted (1) in the stunnel log. I have set up all the iptables settings required in the stunnel docs and made sure that everything is running as root.
I've been working on this problem for three days now and made all the variations on my search terms purple, to no avail. I'm hoping here lies a server wizard to help me with my troubles.
stunnel doc reference: https://www.stunnel.org/static/stunnel.html#SERVICE-LEVEL-OPTIONS
The Setup:
Server is running Ubuntu 12.04.5 LTS (GNU/Linux 2.6.32-042stab093.4 x86_64)
Using stunnel 4
The open_server.php file:
use Ratchet\App;
use Websocket_Server\Server;
require dirname(__DIR__) . '/vendor/autoload.php';
$loop = React\EventLoop\Factory::create();
$webSock = new React\Socket\Server($loop);
$webSock->listen(8888, '0.0.0.0');
$webServer = new Ratchet\Server\IoServer(
new Ratchet\Http\HttpServer(
new Ratchet\WebSocket\WsServer(new Server($loop))
), $webSock
);
$loop->run();
The stunnel.conf:
key = <key file dir>
cert = <crt file dir>
debug = 7
output = /var/log/stunnel_log.log
setgid = 0
[websocket]
accept = 32770
connect = 8888
transparent = source
The entire debug output when trying to hit the socket with transparent = source:
2015.07.26 15:09:26 LOG7[14108:140701658388224]: local socket: FD=0 allocated (non-blocking mode)
2015.07.26 15:09:26 LOG7[14108:140701658388224]: Service websocket accepted FD=0 from <MY EXTERNAL IP>:54421
2015.07.26 15:09:26 LOG7[14108:140701658478336]: Service websocket started
2015.07.26 15:09:26 LOG7[14108:140701658478336]: Option TCP_NODELAY set on local socket
2015.07.26 15:09:26 LOG7[14108:140701658478336]: Waiting for a libwrap process
2015.07.26 15:09:26 LOG7[14108:140701658478336]: Acquired libwrap process #0
2015.07.26 15:09:26 LOG7[14108:140701658478336]: Releasing libwrap process #0
2015.07.26 15:09:26 LOG7[14108:140701658478336]: Released libwrap process #0
2015.07.26 15:09:26 LOG7[14108:140701658478336]: Service websocket permitted by libwrap from <MY EXTERNAL IP>:54421
2015.07.26 15:09:26 LOG5[14108:140701658478336]: Service websocket accepted connection from <MY EXTERNAL IP>:54421
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): before/accept initialization
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 read client hello A
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 write server hello A
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 write certificate A
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 write key exchange A
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 write server done A
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 flush data
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 read client key exchange A
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 read finished A
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 write session ticket A
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 write change cipher spec A
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 write finished A
2015.07.26 15:09:26 LOG7[14108:140701658478336]: SSL state (accept): SSLv3 flush data
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 0 items in the session cache
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 0 client connects (SSL_connect())
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 0 client connects that finished
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 0 client renegotiations requested
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 1 server connects (SSL_accept())
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 1 server connects that finished
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 0 server renegotiations requested
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 0 session cache hits
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 0 external session cache hits
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 0 session cache misses
2015.07.26 15:09:26 LOG7[14108:140701658478336]: 0 session cache timeouts
2015.07.26 15:09:26 LOG6[14108:140701658478336]: SSL accepted: new session negotiated
2015.07.26 15:09:26 LOG6[14108:140701658478336]: Negotiated ciphers: ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
2015.07.26 15:09:26 LOG7[14108:140701658478336]: remote socket: FD=1 allocated (non-blocking mode)
2015.07.26 15:09:26 LOG3[14108:140701658478336]: setsockopt IP_TRANSPARENT: Operation not permitted (1)
2015.07.26 15:09:26 LOG3[14108:140701658478336]: local_bind (original port): Cannot assign requested address (99)
2015.07.26 15:09:26 LOG5[14108:140701658478336]: Connection reset: 0 bytes sent to SSL, 0 bytes sent to socket
2015.07.26 15:09:26 LOG7[14108:140701658478336]: Service websocket finished (0 left)
2015.07.26 15:09:26 LOG7[14108:140701658478336]: str_stats: 0 block(s), 0 byte(s)
Bash output when checking permission:
# ps aux | grep stunnel4
root 14103 0.0 0.1 29820 1032 pts/2 S 15:09 0:00 /usr/bin/stunnel4 /etc/stunnel/stunnel.conf
root 14104 0.0 0.1 29820 704 pts/2 S 15:09 0:00 /usr/bin/stunnel4 /etc/stunnel/stunnel.conf
root 14105 0.0 0.1 29820 704 pts/2 S 15:09 0:00 /usr/bin/stunnel4 /etc/stunnel/stunnel.conf
root 14106 0.0 0.1 29820 704 pts/2 S 15:09 0:00 /usr/bin/stunnel4 /etc/stunnel/stunnel.conf
root 14107 0.0 0.1 29820 704 pts/2 S 15:09 0:00 /usr/bin/stunnel4 /etc/stunnel/stunnel.conf
root 14108 0.0 0.4 95424 2252 ? Ss 15:09 0:00 /usr/bin/stunnel4 /etc/stunnel/stunnel.conf
root 20477 0.0 0.1 6460 776 pts/5 S+ 17:00 0:00 grep --color=auto stunnel4
