1

I have a systemd unit intended to establish a SSH tunnel between two servers. The server which has the unit runs Debian 9. This is what the .service file looks like, except for a few Documentation directives which I have elided here for brevity (they are not the problem, and they are parsed just fine by systemd):

# cat /etc/systemd/system/ssh-tunnel-remote1.service
[Unit]
Description=SSH tunnel for services on remote1
After=network-online.target
[Install]
WantedBy=networking.target
[Service]
Type=simple
User=ssh-remote1
Group=ssh-remote1
Environment=AUTOSSH_POLL=90
ExecStart=/usr/bin/autossh -M 0 -q -N -p 15539 -o "PubkeyAuthentication yes" -o "PreferredAuthentications publickey" -o "IdentityFile /home/ssh-remote1/.ssh/id_rsa" -L 9999:127.0.0.1:X ssh-tunnel@remote1.example.com
Restart=always
PrivateTmp=true
#

(Note: the X in the -L is a real port number.)

On the server where this service is being run, /usr/bin is on /, so it's not an issue of the file system not being mounted when the service is being started.

The After=network-online.target should be plenty enough for DNS to be available, and even if that was the problem, you'd think that systemd would restart the service when it fails.

The service itself looks like it is enabled:

# find /etc/systemd -name ssh-tunnel-remote1\*
/etc/systemd/system/networking.target.wants/ssh-tunnel-remote1.service
/etc/systemd/system/ssh-tunnel-remote1.service
#

but systemctl list-units doesn't seem to know about it:

# systemctl list-units -t service --all | grep ssh-tunnel-remote1
#

I have tried various permutations of systemctl daemon-reload, systemctl reenable ssh-tunnel-remote1, systemctl enable ssh-tunnel-remote1, systemctl disable ssh-tunnel-remote1 and reboot.

Seemingly no matter what I do, after booting, the service shows up as inactive (dead):

# systemctl -o verbose  -l status ssh-tunnel-remote1
● ssh-tunnel-remote1.service - SSH tunnel for services on remote1
   Loaded: loaded (/etc/systemd/system/ssh-tunnel-remote1.service; enabled; vendor preset: enabled)
   Active: inactive (dead)
#

However, it starts just fine if I do it manually:

# systemctl start ssh-tunnel-remote1
# systemctl status ssh-tunnel-remote1
● ssh-tunnel-remote1.service - SSH tunnel for services on remote1
   Loaded: loaded (/etc/systemd/system/ssh-tunnel-remote1.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2017-07-10 13:01:11 UTC; 55s ago
 Main PID: 17835 (autossh)
    Tasks: 2 (limit: 4915)
   CGroup: /system.slice/ssh-tunnel-remote1.service
           ├─17835 /usr/lib/autossh/autossh -M 0 -q -N -p 15539 -o PubkeyAuthentication yes -o PreferredAuthentications publickey -o IdentityFile /home/ssh-remote1/.ssh/id_rsa -L 9999:127.0.0.1:X ssh-tunnel
           └─17838 /usr/bin/ssh -q -N -p 15539 -o PubkeyAuthentication yes -o PreferredAuthentications publickey -o IdentityFile /home/ssh-remote1/.ssh/id_rsa -L 9999:127.0.0.1:X ssh-tunnel@remote1.example.

Jul 10 13:01:11 localhost systemd[1]: Started SSH tunnel for services on remote1.
Jul 10 13:01:11 localhost autossh[17835]: port set to 0, monitoring disabled
Jul 10 13:01:11 localhost autossh[17835]: starting ssh (count 1)
Jul 10 13:01:11 localhost autossh[17835]: ssh child pid is 17838
# telnet 127.0.0.1 9999
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
<usable connection here>
Connection closed by foreign host.
#

Immediately after a reboot, journalctl -xru ssh-tunnel-remote1.service simply prints -- No entries --. Searching manually through the output of journalctl also doesn't show it anywhere at all. In contrast, after starting the service manually, the same command outputs something very similar to:

-- Logs begin at Mon 2017-07-10 12:46:14 UTC, end at Mon 2017-07-10 13:10:24 UTC. --
Jul 10 13:01:11 localhost autossh[17835]: ssh child pid is 17838
Jul 10 13:01:11 localhost autossh[17835]: starting ssh (count 1)
Jul 10 13:01:11 localhost autossh[17835]: port set to 0, monitoring disabled
Jul 10 13:01:11 localhost systemd[1]: Started SSH tunnel for services on remote1.
-- Subject: Unit ssh-tunnel-remote1.service has finished start-up
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- Unit ssh-tunnel-remote1.service has finished starting up.
-- 
-- The start-up result is done.

This is a homegrown .service file, but it's working fine on another server that runs Debian 8.

I have tried placing it both under /etc/systemd/system and /lib/systemd/system, with no apparent difference.

When executed from the command line as su -l ssh-remote1 -c '/usr/bin/autossh -M 0 -q ...', autossh and ssh runs fine in the foreground and the tunnel is available.

I'm practically certain that I am missing some simple difference between Debian 9's systemd 232 and Debian 8's systemd 215, but what? What incantations are required to make that service start on boot on Debian 9?

user
  • 4,335
  • 4
  • 34
  • 71
  • Does it work if you change `networking.target` to `network.target`? `networking.target` is not a target on my Debian 9 system. – jordanm Jul 10 '17 at 13:35
  • Actually, you probably want `WantedBy=multi-user.target` and `After=network.target`. – jordanm Jul 10 '17 at 13:39
  • @jordanm That's making progress. At least now it shows up in `journalctl` output after reboot, albeit as an error; now systemd is complaining that it is "deleted to break ordering cycle starting with network.target/start". – user Jul 10 '17 at 13:41
  • @jordanm That did it! From a glance at the variant on the Debian 8 server, it looks like they changed from `networking.target` (on Debian 8) to `network.target` (on Debian 9). If you write your comments up as a short answer, I'll happily accept it. Thank you for the pointer! – user Jul 10 '17 at 13:44
  • 1
    Just in case a summary helps anyone anywhen (because in all my searching I failed to find anything that actually helped me fix this), on Debian 9, setting `After=network.target` and `WantedBy=multi-user.target` caused the service to start just fine on boot. – user Jul 10 '17 at 13:53

0 Answers0