autossh not working for two or more tunnels - or is there an alternative?

Question

I am trying to use a SSH server as a gateway to connect to more than one internal servers. Internal in this context means that they are not accessible directly, they got no public IP assigned to them.

So the scenario should look like this (example with 2 servers, might be more) with the gateways public IP 123.456.789.45 the internal being 10.12.40.13

+--------+                 +---------+                +----------+
| client |--> 2214/tcp --> |         | --> 22/tcp --> | Server 1 |
+--------+                 |         |                +----------+
                           | Gateway |
+--------+                 |         |                +----------+
| client |--> 2215/tcp --> |         | --> 22/tcp --> | Server 2 |
+--------+                 +---------+                +----------+

My first approach was to set them up from the gateway to the servers with something like

ssh -N -L 123.456.789.45:2214:127.0.0.1:22 tunnel-user@server1
ssh -N -L 123.456.789.45:2215:127.0.0.1:22 tunnel-user@server2

While that works I stumbled upon the problem of the tunnels not being too reliable, failing every here and there. The logical next step was trying to get autossh running. And here I got a bunch of problems. The first tunnel can be established without problems using

autossh -M 20000 -f -N -L 123.456.789.45:2214:127.0.0.1:22 tunnel-user@server1

I can get access to server1 by outside connecting to the gateway at port 2214. However I can't get the second one up and running with autossh. Headbanging a couple of hours now I decided to try it vice versa. So:

The second approach was to set them up from the servers to the gateway. Again while the variant with pure ssh works using something like this ...

ssh -R 123.456.789.45:2214:127.0.0.1:22 tunnel-user@gateway # <- init from server 1
ssh -R 123.456.789.45:2215:127.0.0.1:22 tunnel-user@gateway # <- init from server 2

... using autossh fails.

autossh -M 20000 -f -R 123.456.789.45:2214:127.0.0.1:22 tunnel-user@gateway

The logfiles simply says nothing. Syslog at least comes up with

ssh exited prematurely with status 0; autossh exiting

Now does anyone know how to solve the autossh issue on either approach? Is there something similar to autossh that I can give a shot? Is there a way to maybe achieve something like a refresh on the pure ssh version mentioned above?

All involved servers are running the latest updates on Ubuntu 10.04 LTS and autossh 1.4b

Does the tunnel need to be persistent, if not, have you thought of using Xinetd? — Ben Lessani, Mar 15 '12 at 22:36
Every tunnel in my scenario needs to be persistent, yes. However how would you try to use xinetd for this? — Chris, Mar 15 '12 at 22:43
It wouldn't be too useful if the connections need to be persistent. In that case, I would **strongly** suggest using a VPN. On Linux OpenVPN SSL tunnels are quite easy to configure and are secure, fast and stable. — Ben Lessani, Mar 15 '12 at 22:46
Are you using "-M 20000" for both tunnels on the same machine? It's not clear from your description, but if so, that may be the problem. The monitoring port and the port above it need to be available. So if you use "-M 20000" for the first tunnel, then maybe use "-M 20002" for the second one. — , Jul 10 '13 at 14:54

bhelm · Answer 1 · 2014-01-21T10:34:36.203

From the autossh documentation:

autossh uses ssh to construct a loop of ssh forwardings (one from local to remote, one from remote to local), and then sends test data that it expects to get back.

-M port[:echo_port] specifies the base monitoring port to use. Without the echo port, this port and the port immediately above it ( port + 1) should be something nothing else is using. autossh will send test data on the base monitoring port, and receive it back on the port above. For example, if you specify "-M 20000", autossh will set up forwards so that it can send data on port 20000 and receive it back on 20001.

if you are using -M 20000 twice, this must fail. Use different ports for that (with one port space between them, so -M 20000 and -M 20002 would work). I recommend doing a "man autossh" and read the documentation of autossh, its also available online: http://www.manpagez.com/man/1/autossh/ . If you are using a lot of autossh tunnels, you may setup a dedicated echo service (From autossh documentation again):

Alternatively, a port for a remote echo service may be specified. This should be port 7 if you wish to use the standard inetd echo service. When an echo port is specified, only the specified monitor port is used, and it carries the monitor message in both directions. This allows the autossh to verify the connection without blocking ports for each tunnel on the remote side.

If you want to use xinetd for that, here is my echo service decleration:

service echo
{
        flags                   = REUSE
        socket_type             = stream
        wait                    = no
        user                    = root
        server                  = /usr/bin/cat
        log_on_failure          += USERID
        only_from               = 127.0.0.1
        disable                 = no
}

then you can use -M 20000:7 on all tunnels from different machines. if you have multiple tunnelns on one machine, use multiple -L or -R options or use a different port like -M 20002:7

score 2 · Answer 2 · answered Aug 27 '13 at 18:51

You can specify both tunnels on the same ssh command.

ssh -R 123.456.789.45:2214:127.0.0.1:22 -R 123.456.789.45:2215:127.0.0.1:22 tunnel-user@gateway

Or may you can try to add tunnels in a .ssh/config like this, so the command line didn't get too crowded:

host server1
        RemoteForward 123.456.789.45:2214:127.0.0.1:22
        RemoteForward 123.456.789.45:2215:127.0.0.1:22

score 1 · Answer 3 · answered Aug 11 '17 at 12:56

You can configure multiple tunnels in autossh's config file. Unfortunately it is not too well documented. For two tunnels, with your given details, and based on PubkeyAuthentication I'd do it like so (SuSE 11 SP 4):

    In /etc/sysconfig/autossh

    # Number of autossh instances to spawn on start.
    AUTOSSH_SPAWNS="3"

    # All options except for the first must end with "_<number>"

    AUTOSSH_OPTIONS_1="tunnel-user1@server1 \
    -i /home/tunnel-user1/.ssh/id_rsa \
    -M 0 -f -N -L2214:127.0.0.1:22 -o ExitOnForwardFailure=yes \
    -o ServerAliveInterval=60 -o ServerAliveCountMax=3 
    -o StrictHostKeyChecking=no"

    AUTOSSH_OPTIONS_2="tunnel-user2@server2 \
    -i /home/tunnel-user2/.ssh/id_rsa \
    -M 0 -f -N -L2215:127.0.0.1:22 -o ExitOnForwardFailure=yes \
    -o ServerAliveInterval=60 -o ServerAliveCountMax=3
    -o StrictHostKeyChecking=no"

Of course everything else with respect to a successful ssh connection with keys must be in place

pub keys of tunnel-users in the servers'
/home/tunnel-user[1|2]/.ssh/authorized_keys files
tunnel-user must exist on the gateways and servers
and be configured in /etc/ssh/sshd_config
on the gateway's sshd_config AllowTcpForwarding must be set yes as well as PermitTunnel

Hi, sorry, but can't find how actually to run autossh with this config file? — Kostanos, Jul 31 '18 at 00:54

score 0 · Accepted Answer · answered Mar 24 '12 at 13:30

I used a different approach to solve this issue. At first I managed to start the SSH tunnels from the clients at boot time. I wrote an init.d script for that, made it a service and let puppet handle it being up.

However this was too much of a hassle and I decided to turn around and use NAT and port forwarding over the gateway server and its UFW configuration. Even tho it isn't an answer to the SSH problem itself, here the solution as it fixed the basic problem:

In /etc/ufw/sysctl enabled net/ipv4/ip_forward=1 and ran sysctl -p

In /etc/ufw/before.rules

*nat
:POSTROUTING ACCEPT [0:0]

# forward traffic from eth1 through eth0
-A POSTROUTING -s 10.12.40.0/24 -o eth0 -j MASQUERADE

# some DNAT rules
-A PREROUTING -i eth0 -p tcp --dport 2214 -j DNAT --to-destination 10.12.40.14:22
-A PREROUTING -i eth0 -p tcp --dport 2215 -j DNAT --to-destination 10.12.40.15:22

#
COMMIT

and ran ufw disable && ufw enable. All needed than was to check the correct routing on the client.

Different approach, wanted result. Thanks everybody for reading and thinking about my problem and @sonassi: I'll give your suggestion with OpenVPN a try the moment I got a bit more time and the requirement for more than the SSH port.

score 0 · Answer 5 · answered Jul 16 '13 at 10:53

don't know about autossh ....

but why don't you try the ssh config option ProxyCommand? (http://undeadly.org/cgi?action=article&sid=20070925181947)

you would need to configure the .ssh/config in the clients like this:

Host Server1
    HostName Server1
    ProxyCommand ssh gateway:2214 nc %h %p 2> /dev/null

good luck!

autossh not working for two or more tunnels - or is there an alternative?

5 Answers5