55

I found this systemd service file to start autossh to keep up a ssh tunnel: https://gist.github.com/thomasfr/9707568

[Unit]
Description=Keeps a tunnel to 'remote.example.com' open
After=network.target

[Service]
User=autossh
# -p [PORT]
# -l [user]
# -M 0 --> no monitoring
# -N Just open the connection and do nothing (not interactive)
# LOCALPORT:IP_ON_EXAMPLE_COM:PORT_ON_EXAMPLE_COM
ExecStart=/usr/bin/autossh -M 0 -N -q -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -p 22 -l autossh remote.example.com -L 7474:127.0.0.1:7474 -i /home/autossh/.ssh/id_rsa

[Install]
WantedBy=multi-user.target

Is there a way to configure systemd to start several tunnels in one service.

I don't want to create N system service files, since I want to avoid copy+paste.

All service files would be identical except "remote.example.com" would be replace with other host names.

1.5 year later ...

I asked this question roughly 1.5 year ago.

My mind has changed. Yes, it's nice, that you can do this with systemd, but I will use configuration-management in the future.

Why should systemd implement a template language and substitute %h? .. I think it makes no sense.

Several months later I think this looping and templating should be solved on a different level. I would use Ansible or TerraForm for this now.

4
  • In other words, you're saying use a configuration management system to generate multiple almost identical service files to accomplish this task? Hmmm, maybe. As with most such matters, there's not a clear dividing line separating these.
    – pgoetz
    Jan 12, 2018 at 17:46
  • @pgoetz config management is still new to me, but it has a benefit if you look at the topic of this question: If you look at the result of the config managment everybody who knows systemd service files will understand it:plain and simple service files. I think it makes more sense to learn and use a config management system since the knowledge can be used for all config in /etc, not just systemd.
    – guettli
    Jan 14, 2018 at 10:03
  • I resorted to using Supervisor supervisord.org to define multiple processes in one file. Some repetition there stays, that's how I arrived here.
    – fmalina
    May 8, 2020 at 18:46
  • I wrote a SystemD service for SSH tunnels github.com/yurt-page/sshtunnel Feb 18 at 16:43

4 Answers 4

60

Well, assuming that the only thing changing per unit file is the remote.example.com part, you can use an Instantiated Service.

From the systemd.unit man page:

Optionally, units may be instantiated from a template file at runtime. This allows creation of multiple units from a single configuration file. If systemd looks for a unit configuration file, it will first search for the literal unit name in the file system. If that yields no success and the unit name contains an "@" character, systemd will look for a unit template that shares the same name but with the instance string (i.e. the part between the "@" character and the suffix) removed. Example: if a service [email protected] is requested and no file by that name is found, systemd will look for [email protected] and instantiate a service from that configuration file if it is found.

Basically, you create a single unit file, which contains a variable (usually %i) where the differences occur and then they get linked when you "enable" that service.

For example, I have a unit file called /etc/systemd/system/[email protected] that looks like this:

[Unit]
Description=AutoSSH service for ServiceABC on %i
After=network.target

[Service]
Environment=AUTOSSH_GATETIME=30 AUTOSSH_LOGFILE=/var/log/autossh/%i.log AUTOSSH_PIDFILE=/var/run/autossh.%i.pid
PIDFile=/var/run/autossh.%i.pid
#Type=forking
ExecStart=/usr/bin/autossh -M 40000 -NR 5000:127.0.0.1:5000 -i /opt/ServiceABC/.ssh/id_rsa_ServiceABC -l ServiceABC %i

[Install]
WantedBy=multi-user.target

Which I've then enabled

[user@anotherhost ~]$ sudo systemctl enable [email protected]
ln -s '/etc/systemd/system/[email protected]' '/etc/systemd/system/multi-user.target.wants/[email protected]'

And can intereact with

[user@anotherhost ~]$ sudo systemctl start [email protected]
[user@anotherhost ~]$ sudo systemctl status [email protected]
[email protected] - AutoSSH service for ServiceABC on somehost.example
   Loaded: loaded (/etc/systemd/system/[email protected]; enabled)
   Active: active (running) since Tue 2015-10-20 13:19:01 EDT; 17s ago
 Main PID: 32524 (autossh)
   CGroup: /system.slice/system-autossh.slice/[email protected]
           ├─32524 /usr/bin/autossh -M 40000 -NR 5000:127.0.0.1:5000 -i /opt/ServiceABC/.ssh/id_rsa_ServiceABC -l ServiceABC somehost.example.com
           └─32525 /usr/bin/ssh -L 40000:127.0.0.1:40000 -R 40000:127.0.0.1:40001 -NR 5000:127.0.0.1:5000 -i /opt/ServiceABC/.ssh/id_rsa_ServiceABC -l ServiceABC somehost.example.com

Oct 20 13:19:01 anotherhost.example.com systemd[1]: Started AutoSSH service for ServiceABC on somehost.example.com.
[user@anotherhost ~]$ sudo systemctl status [email protected]
[user@anotherhost ~]$ sudo systemctl status [email protected]
[email protected] - AutoSSH service for ServiceABC on somehost.example.com
   Loaded: loaded (/etc/systemd/system/[email protected]; enabled)
   Active: inactive (dead) since Tue 2015-10-20 13:24:10 EDT; 2s ago
  Process: 32524 ExecStart=/usr/bin/autossh -M 40000 -NR 5000:127.0.0.1:5000 -i /opt/ServiceABC/.ssh/id_rsa_ServiceABC -l ServiceABC %i (code=exited, status=0/SUCCESS)
 Main PID: 32524 (code=exited, status=0/SUCCESS)

Oct 20 13:19:01 anotherhost.example.com systemd[1]: Started AutoSSH service for ServiceABC on somehost.example.com.
Oct 20 13:24:10 anotherhost.example.com systemd[1]: Stopping AutoSSH service for ServiceABC on somehost.example.com...
Oct 20 13:24:10 anotherhost.example.com systemd[1]: Stopped AutoSSH service for ServiceABC on somehost.example.com.

As you can see, all instances of %i in the unit file get replaced with somehost.example.com.

There's a bunch more specifiers that you can use in a unit file though, but I find %i to work best in cases like this.

6
  • 2
    Wow, systemd is great.
    – guettli
    Oct 21, 2015 at 7:33
  • You don't show how to start automatically at boot, including of which ones to start. Jul 12, 2018 at 9:56
  • With Systemd, the enable action is what makes a unit/service start at boot.
    – GregL
    Jul 12, 2018 at 12:08
  • Can I independently enable/disable the instances? Oct 1, 2018 at 12:12
  • Yeah, that's what you're doing when you enable/disable them.
    – GregL
    Oct 1, 2018 at 12:13
32

Here is a python example, which was what I was looking for. The @ in the service filename lets you start N processes:

$ cat /etc/systemd/system/[email protected]

[Unit]
Description=manages my worker service, instance %i
After=multi-user.target

[Service]
PermissionsStartOnly=true
Type=idle
User=root
ExecStart=/usr/local/virtualenvs/bin/python /path/to/my/script.py
Restart=always
TimeoutStartSec=10
RestartSec=10

Various methods to call it

Enabling various counts for example:

  • Enable 30 workers:

    sudo systemctl enable my-worker\@{1..30}.service
    
  • Enable 2 workers:

    sudo systemctl enable my-worker\@{1..2}.service
    

Then be sure to reload:

sudo systemctl daemon-reload

Now you can start/stop then in various ways:

  • Start 1:

    sudo systemctl start [email protected]
    
  • Start Multiple:

    sudo systemctl start my-worker@{1..2}
    
  • Stop Multiple:

    sudo systemctl stop my-worker@{1..2}
    
  • Check status:

    sudo systemctl status my-worker@1
    

UPDATE: To manage instances as one service, you can do something like this:

/etc/systemd/system/[email protected]:

[Unit]
Description=manage worker instances as a service, instance %i
Requires=some-worker.service
Before=some-worker.service
BindsTo=some-worker.service

[Service]
PermissionsStartOnly=true
Type=idle
User=root
#EnvironmentFile=/etc/profile.d/optional_envvars.sh
ExecStart=/usr/local/virtualenvs/bin/python /path/to/my/script.py
TimeoutStartSec=10
RestartSec=10

[Install]
WantedBy=some-worker.service

/usr/bin/some-worker-start.sh:

#!/bin/bash
systemctl start some-worker@{1..10}

/etc/systemd/system/some-worker.service:

[Unit]
Description=manages some worker instances as a service, instance

[Service]
Type=oneshot
ExecStart=/usr/bin/sh /usr/bin/some-worker-start.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

And now you can manage all instances with sudo systemctl some-worker (start|restart|stop)

Here is some boilerplate for your script.py:

#!/usr/bin/env python

import logging


def worker_loop():
    shutdown = False
    while True:

        try:
            if shutdown:
                break

            # Your execution logic here.
            # Common logic - i.e. consume from a queue, perform some work, ack message
            print("hello world")

        except (IOError, KeyboardInterrupt):
            shutdown = True
            logging.info("shutdown received - processing will halt when jobs complete")
        except Exception as e:
            logging.exception("unhandled exception on shutdown. {}".format(e))


if __name__ == '__main__':
    worker_loop()
8
  • @radek: Two things I do not understand: First, %i is only used in the description of the unit file. How does the start command know what to start? Second, how does systemctl some-worker (start|restart|stop) know which instances to work on?
    – U. Windl
    Mar 20, 2019 at 13:31
  • %i is the output from @ in the name of the service file. Second part is already explained in the answer, see Now you can start/stop then in various ways.
    – radtek
    Mar 20, 2019 at 20:03
  • I think his answer is incomplete without the scripts being involved. Most "magic" is done inside the scripts which are missing.
    – U. Windl
    May 7, 2019 at 12:43
  • I have provided a full working solution here actually. Which "scripts" are you referring to? /path/to/my/script.py can be whatever you want, a "hello world" if you want. Something that will stay up running until it receives a kill signal. Please note the question is not specific to python.
    – radtek
    May 8, 2019 at 14:36
  • Wow it lets you start multiples at a time? mind blown...
    – rogerdpack
    Jun 24, 2019 at 13:58
2

GregL's answer helped me a great deal. Here is an example of a unit template I used in my code using the example above for a gearman job server. I made a shell script that lets me create X amount of "workers" using this one template.

[Unit]
Description=az gearman worker
After=gearman-job-server.service

[Service]
PIDFile=/var/run/gearman_worker_az%i.pid
Type=simple
User=www-data
WorkingDirectory=/var/www/mysite.com/jobs/
ExecStart=/usr/bin/php -f gearman_worker_az.php > /dev/null 2>&1
Restart=on-success
KillMode=process

[Install]
WantedBy=multi-user.target
0
-1

I have searched for solution to a similar task and actually have found one, that i believe is easier to accomplish, but is supposed to be hacky. (and is not mentioned here)

I needed to create multiple ssh connections after vpn creates a tunnel, so i have created a service that is dependent on tun device and calls shell script with appropriate commands.

Service /etc/systemd/system/ssh_tunnel.service:

[Unit]
Description=Reverse SSH Service to access hidden services
ConditionPathExists=|/usr/bin
Wants=sys-devices-virtual-net-tun0.device
After=network.target sys-devices-virtual-net-tun0.device

[Service]
Type=forking
ExecStart=/bin/sh /etc/openvpn/ssh_tunnels.sh 
RemainAfterExit=yes
TimeoutSec=0
GuessMainPID=no

[Install]
WantedBy=multi-user.target

/etc/openvpn/ssh_tunnels.sh:

!/bin/bash
#sleep 15

echo 'Tunelling some ports'
killall -HUP ssh

su - user -c 'ssh -f [email protected] -p 9999 -L :3690:svn.newbox.ru:3690 -L :8888:10.1.20.55:80 -L :8181:10.1.10.10:80 -N -vvv'

ssh -i /home/user/.ssh/id_rsa -f [email protected] -p 9999 -L :587:mail.domain.ru:587 -L :995:mail.newbox.ru:995 -L :22:10.1.2.1:22 -N -vvv &

exit 0

Result:

# systemctl status ssh_tunnel.service
● ssh_tunnel.service - Reverse SSH Service to access hidden services
     Loaded: loaded (/etc/systemd/system/ssh_tunnel.service; enabled; vendor preset: disabled)
     Active: active (running) since Fri 2020-03-20 16:01:07 UTC; 22min ago
    Process: 156 ExecStart=/bin/sh /etc/openvpn/ssh_tunnel.sh (code=exited, status=0/SUC>
      Tasks: 2 (limit: 4915)
     Memory: 3.8M
     CGroup: /system.slice/ssh_tunnel.service
             ├─166 ssh -f [email protected] -p 9999 -L :3690:svn.newbox.ru:3690 -L :8888:10.1.20.55:80 ->
             └─168 ssh -i /home/user/.ssh/id_rsa -f [email protected] -p 9999 -L :587:mail.newbox.ru:5>
...

However, i haven't checked yet, how it survives vpn restart, but it's another topic.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .