SSH tunnel to Docker container

Question

Goal: connect locally to a remote repl (e.g. via lein repl :connect).

Locally, this is easy:

1. Run server (it starts an embedded nrepl server on port 8081)
2. Run lein repl :connect 8081 & voila! repl connected

I've also done this to connect to a repl on a remote server when the repl was running on a port that was not open, by using a SSH tunnel:

1. On some.host, run server (it starts embedded nrepl server on port 8081)
2. SSH tunnel ssh -N -T -L 8081:localhost:8081 me@some.host
3. Locally, lein repl :connect 8081 & voila! repl connected

However, my current setup is that "server" is run in a Docker container, which maps port 8081. So, in order to connect to the nrepl server, it's gotta go local -> some.host -> docker-container -> nrepl.

I can see that my docker container has port 8081 mapped:

$ sudo docker port container-id 8081
0.0.0.0:8081

And, on the server hosting the docker container, I can see that port 8081 is listening:

$ netstat -anl | sed -n '2p;/8081/p'
Proto    Recv-Q    Send-Q    Local Address   Foreign Address     State  
tcp      0         0         :::8081         :::*                LISTEN

And it seems like I can open a SSH tunnel for port 8081; e.g. no errors/warnings from running:

ssh -N -T -L *:8081:localhost:8081 me@some.host

Which makes me think that I have the correct SSH tunnel, except that whenever I try to connect to the running repl server, it immediately fails, like so:

$ lein repl :connect 8081
Connecting to nREPL at 127.0.0.1:8081
SocketException The transport's socket appears to have lost its connection to the nREPL server

It's notable that the error is the connection is lost, because run without the SSH tunnel open, the same command fails with Connection refused. That makes me think that the SSH tunnel is OK and that the problem is on the server forwarding to the docker container, which is why the title of this is a generic question about opening a tunnel from client -> server -> docker container.

I thought it might be something to do with SSH GatewayPorts, so I tried enabling GatewayPorts but that didn't change anything.

Questions:

1. Is there anything obviously wrong with the SSH tunnel approach above?
2. How can I determine where the connection is being dropped?
3. Any other suggestions?

Thanks!

Answer 1

And... it turns out there was nothing wrong with the tunnel.

The problem was that starting an embedded nrepl server inside a docker container requires an explicit binding to "0.0.0.0". Without that, the nrepl server by default binds to "localhost", which makes it inaccessible from outside the docker container, despite the port mapping, because the docker port mapping uses 0.0.0.0:. Once the nrepl server was bound correctly, the port mapping took care of the rest.

Answer 2

Just as a helpful link to have a look at - Docker SSH. Very simplified way of managing SSH for Docker.