36

I found this systemd service file to start autossh to keep up a ssh tunnel: https://gist.github.com/thomasfr/9707568

[Unit]
Description=Keeps a tunnel to 'remote.example.com' open
After=network.target

[Service]
User=autossh
# -p [PORT]
# -l [user]
# -M 0 --> no monitoring
# -N Just open the connection and do nothing (not interactive)
# LOCALPORT:IP_ON_EXAMPLE_COM:PORT_ON_EXAMPLE_COM
ExecStart=/usr/bin/autossh -M 0 -N -q -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -p 22 -l autossh remote.example.com -L 7474:127.0.0.1:7474 -i /home/autossh/.ssh/id_rsa

[Install]
WantedBy=multi-user.target

Is there a way to configure systemd to start several tunnels in one service.

I don't want to create N system service files, since I want to avoid copy+paste.

All service files would be identical except "remote.example.com" would be replace with other host names.

1.5 year later ...

I asked this question roughly 1.5 year ago.

My mind has changed a bit. Yes, it's nice, that you can do this with systemd (I still use it), but I will use configuration-management in the future.

Why should systemd implement a template language and substitute %h?

Several months later I think this loop and templating should be solved with a tool which automates configuration. I use one tool of this list at wikipedia now.

  • In other words, you're saying use a configuration management system to generate multiple almost identical service files to accomplish this task? Hmmm, maybe. As with most such matters, there's not a clear dividing line separating these. – pgoetz Jan 12 '18 at 17:46
  • @pgoetz config management is still new to me, but it has a benefit if you look at the topic of this question: If you look at the result of the config managment everybody who knows systemd service files will understand it:plain and simple service files. I think it makes more sense to learn and use a config management system since the knowledge can be used for all config in /etc, not just systemd. – guettli Jan 14 '18 at 10:03
47

Well, assuming that the only thing changing per unit file is the remote.example.com part, you can use an Instantiated Service.

From the systemd.unit man page:

Optionally, units may be instantiated from a template file at runtime. This allows creation of multiple units from a single configuration file. If systemd looks for a unit configuration file, it will first search for the literal unit name in the file system. If that yields no success and the unit name contains an "@" character, systemd will look for a unit template that shares the same name but with the instance string (i.e. the part between the "@" character and the suffix) removed. Example: if a service getty@tty3.service is requested and no file by that name is found, systemd will look for getty@.service and instantiate a service from that configuration file if it is found.

Basically, you create a single unit file, which contains a variable (usually %i) where the differences occur and then they get linked when you "enable" that service.

For example, I have a unit file called /etc/systemd/system/autossh@.service that looks like this:

[Unit]
Description=AutoSSH service for ServiceABC on %i
After=network.target

[Service]
Environment=AUTOSSH_GATETIME=30 AUTOSSH_LOGFILE=/var/log/autossh/%i.log AUTOSSH_PIDFILE=/var/run/autossh.%i.pid
PIDFile=/var/run/autossh.%i.pid
#Type=forking
ExecStart=/usr/bin/autossh -M 40000 -NR 5000:127.0.0.1:5000 -i /opt/ServiceABC/.ssh/id_rsa_ServiceABC -l ServiceABC %i

[Install]
WantedBy=multi-user.target

Which I've then enabled

[user@anotherhost ~]$ sudo systemctl enable autossh@somehost.example.com
ln -s '/etc/systemd/system/autossh@.service' '/etc/systemd/system/multi-user.target.wants/autossh@somehost.example.com.service'

And can intereact with

[user@anotherhost ~]$ sudo systemctl start autossh@somehost.example.com
[user@anotherhost ~]$ sudo systemctl status autossh@somehost.example.com
autossh@somehost.example.service - AutoSSH service for ServiceABC on somehost.example
   Loaded: loaded (/etc/systemd/system/autossh@.service; enabled)
   Active: active (running) since Tue 2015-10-20 13:19:01 EDT; 17s ago
 Main PID: 32524 (autossh)
   CGroup: /system.slice/system-autossh.slice/autossh@somehost.example.com.service
           ├─32524 /usr/bin/autossh -M 40000 -NR 5000:127.0.0.1:5000 -i /opt/ServiceABC/.ssh/id_rsa_ServiceABC -l ServiceABC somehost.example.com
           └─32525 /usr/bin/ssh -L 40000:127.0.0.1:40000 -R 40000:127.0.0.1:40001 -NR 5000:127.0.0.1:5000 -i /opt/ServiceABC/.ssh/id_rsa_ServiceABC -l ServiceABC somehost.example.com

Oct 20 13:19:01 anotherhost.example.com systemd[1]: Started AutoSSH service for ServiceABC on somehost.example.com.
[user@anotherhost ~]$ sudo systemctl status autossh@somehost.example.com
[user@anotherhost ~]$ sudo systemctl status autossh@somehost.example.com
autossh@somehost.example.com.service - AutoSSH service for ServiceABC on somehost.example.com
   Loaded: loaded (/etc/systemd/system/autossh@.service; enabled)
   Active: inactive (dead) since Tue 2015-10-20 13:24:10 EDT; 2s ago
  Process: 32524 ExecStart=/usr/bin/autossh -M 40000 -NR 5000:127.0.0.1:5000 -i /opt/ServiceABC/.ssh/id_rsa_ServiceABC -l ServiceABC %i (code=exited, status=0/SUCCESS)
 Main PID: 32524 (code=exited, status=0/SUCCESS)

Oct 20 13:19:01 anotherhost.example.com systemd[1]: Started AutoSSH service for ServiceABC on somehost.example.com.
Oct 20 13:24:10 anotherhost.example.com systemd[1]: Stopping AutoSSH service for ServiceABC on somehost.example.com...
Oct 20 13:24:10 anotherhost.example.com systemd[1]: Stopped AutoSSH service for ServiceABC on somehost.example.com.

As you can see, all instances of %i in the unit file get replaced with somehost.example.com.

There's a bunch more specifiers that you can use in a unit file though, but I find %i to work best in cases like this.

  • Wow, systemd is great. – guettli Oct 21 '15 at 7:33
  • You don't show how to start automatically at boot, including of which ones to start. – Craig Hicks Jul 12 '18 at 9:56
  • With Systemd, the enable action is what makes a unit/service start at boot. – GregL Jul 12 '18 at 12:08
  • Can I independently enable/disable the instances? – Soumya Kanti Oct 1 '18 at 12:12
  • Yeah, that's what you're doing when you enable/disable them. – GregL Oct 1 '18 at 12:13
15

Here is a python example, which was what I was looking for. The @ in the service filename lets you start N processes:

$ cat /etc/systemd/system/my-worker@.service

[Unit]
Description=manages my worker service, instance %i
After=multi-user.target

[Service]
PermissionsStartOnly=true
Type=idle
User=root
ExecStart=/usr/local/virtualenvs/bin/python /path/to/my/script.py
Restart=always
TimeoutStartSec=10
RestartSec=10

Various methods to call it

Enabling various counts for example:

  • Enable 30 workers:

    sudo systemctl enable my-worker\@{1..30}.service
    
  • Enable 2 workers:

    sudo systemctl enable my-worker\@{1..2}.service
    

Then be sure to reload:

sudo systemctl daemon-reload

Now you can start/stop then in various ways:

  • Start 1:

    sudo systemctl start my-worker@2.service
    
  • Start Multiple:

    sudo systemctl start my-worker@{1..2}
    
  • Stop Multiple:

    sudo systemctl stop my-worker@{1..2}
    
  • Check status:

    sudo systemctl status my-worker@1
    

UPDATE: To manage instances as one service, you can do something like this:

/etc/systemd/system/some-worker@.service:

[Unit]
Description=manage worker instances as a service, instance %i
Requires=some-worker.service
Before=some-worker.service
BindsTo=some-worker.service

[Service]
PermissionsStartOnly=true
Type=idle
User=root
#EnvironmentFile=/etc/profile.d/optional_envvars.sh
ExecStart=/usr/local/virtualenvs/bin/python /path/to/my/script.py
TimeoutStartSec=10
RestartSec=10

[Install]
WantedBy=some-worker.service

/usr/bin/some-worker-start.sh:

#!/bin/bash
systemctl start some-worker@{1..10}

/etc/systemd/system/some-worker.service:

[Unit]
Description=manages some worker instances as a service, instance

[Service]
Type=oneshot
ExecStart=/usr/bin/sh /usr/bin/some-worker-start.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

And now you can manage all instances with sudo systemctl some-worker (start|restart|stop)

Here is some boilerplate for your script.py:

#!/usr/bin/env python

import logging


def worker_loop():
    shutdown = False
    while True:

        try:
            if shutdown:
                break

            # Your execution logic here.
            # Common logic - i.e. consume from a queue, perform some work, ack message
            print("hello world")

        except (IOError, KeyboardInterrupt):
            shutdown = True
            logging.info("shutdown received - processing will halt when jobs complete")
        except Exception as e:
            logging.exception("unhandled exception on shutdown. {}".format(e))


if __name__ == '__main__':
    worker_loop()
  • @radek: Two things I do not understand: First, %i is only used in the description of the unit file. How does the start command know what to start? Second, how does systemctl some-worker (start|restart|stop) know which instances to work on? – U. Windl Mar 20 at 13:31
  • %i is the output from @ in the name of the service file. Second part is already explained in the answer, see Now you can start/stop then in various ways. – radtek Mar 20 at 20:03
  • I think his answer is incomplete without the scripts being involved. Most "magic" is done inside the scripts which are missing. – U. Windl May 7 at 12:43
  • I have provided a full working solution here actually. Which "scripts" are you referring to? /path/to/my/script.py can be whatever you want, a "hello world" if you want. Something that will stay up running until it receives a kill signal. Please note the question is not specific to python. – radtek May 8 at 14:36
  • Wow it lets you start multiples at a time? mind blown... – rogerdpack Jun 24 at 13:58
1

GregL's answer helped me a great deal. Here is an example of a unit template I used in my code using the example above for a gearman job server. I made a shell script that lets me create X amount of "workers" using this one template.

[Unit]
Description=az gearman worker
After=gearman-job-server.service

[Service]
PIDFile=/var/run/gearman_worker_az%i.pid
Type=simple
User=www-data
WorkingDirectory=/var/www/mysite.com/jobs/
ExecStart=/usr/bin/php -f gearman_worker_az.php > /dev/null 2>&1
Restart=on-success
KillMode=process

[Install]
WantedBy=multi-user.target

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.