0

I used this answer as a guide to help me writer a startup script for systemd to start my Jenkins container when my machine boots up. However, the script is not working. Here is the script:

[Unit]
Description=Docker container that houses the Jenkins build service.
After=network-online.target

[Service]
Group=docker
ExecStart=/usr/bin/docker start jenkins
ExecStop=/usr/bin/docker stop jenkins

[Install]
WantedBy=multi-user.target

I placed it in /etc/systemd/system/jenkins.service.

When I run sudo systemctl start jenkins, nothing happens. No errors or anything printed out, and the container does not start (if I run docker ps, there are no containers listed as running).

I can run /usr/bin/docker start jenkins from the command line manually and it starts perfectly fine, so the issue seems to be in the way I've written the script, but I can't figure out why it's not working as expected. Any help is appreciated.

Update:

I've updated the script by changing the ExecStart and ExecStop values to be /bin/sh -c '/usr/bin/docker start jenkins' and /bin/sh -c '/usr/bin/docker stop jenkins', respectively, but that still didn't work.

Update 2:

I updated the ExecStart and ExecStop by removing the /bin/sh -c "..." and just enclosing the command in double quotes, but now I get an interesting output in sudo systemctl status jenkins:

$ sudo systemctl status jenkins
● jenkins.service - Docker container that houses the Jenkins build service.
   Loaded: loaded (/etc/systemd/system/jenkins.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2017-12-03 12:20:39 EST; 10s ago
  Process: 1513 ExecStop=/usr/bin/docker stop jenkins (code=exited, status=203/EXEC)
  Process: 1511 ExecStart=/usr/bin/docker start jenkins (code=exited, status=203/EXEC)
 Main PID: 1511 (code=exited, status=203/EXEC)

Dec 03 12:20:39 evt-jenkins systemd[1]: Started Docker container that houses the Jenkins build service..
Dec 03 12:20:39 evt-jenkins systemd[1]: jenkins.service: Main process exited, code=exited, status=203/EXEC
Dec 03 12:20:39 evt-jenkins systemd[1]: jenkins.service: Control process exited, code=exited status=203
Dec 03 12:20:39 evt-jenkins systemd[1]: jenkins.service: Unit entered failed state.
Dec 03 12:20:39 evt-jenkins systemd[1]: jenkins.service: Failed with result 'exit-code'.

Is there a way I can see exactly what is failing? These error codes aren't really indicative of anything.

Update 3:

Okay, so I stared from the beginning again and removed the quotes. The jenkins.service file now reads exactly as it does at the top of the question. When I reload systemd, and start the service, this is the output of sudo systemctl status jenkins:

$ sudo systemctl status jenkins
● jenkins.service - Docker container that houses the Jenkins build service.
   Loaded: loaded (/etc/systemd/system/jenkins.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Sun 2017-12-03 12:31:59 EST; 2s ago
  Process: 1683 ExecStop=/usr/bin/docker stop jenkins (code=exited, status=0/SUCCESS)
  Process: 1594 ExecStart=/usr/bin/docker start jenkins (code=exited, status=0/SUCCESS)
 Main PID: 1594 (code=exited, status=0/SUCCESS)

Dec 03 12:31:59 evt-jenkins systemd[1]: Started Docker container that houses the Jenkins build service..
Dec 03 12:31:59 evt-jenkins docker[1594]: jenkins
Dec 03 12:31:59 evt-jenkins docker[1683]: jenkins

So systemd not only reports a successful run of the script, but it also shows the right output of starting a container by its name (rather than its SHA). But when I run docker ps, there is nothing listed!! Might I remind you that when I run the docker command manually, the container runs just fine.

Any thoughts? Systemd is proving itself to be quite difficult to work with.

Update 4:

Okay, so I just noticed something very strange going on here. If you notice in the above snippet (the output of sudo systemctl status jenkins), you'll notice that the PID of the ExecStart is 1594, and the PID of the ExecStop is 1683. Now further down in the output, you'll see that the output of the ExecStart is coming first, before the ExecStop. I'm not sure if this would be because of output buffering, or if systemd is making the ExecStart run before the ExecStop.

  • Usually containers are initially created with docker run whereas docker start is used to start already-created, stopped containers. Does your container already exist? – Patrick Dec 3 '17 at 19:18
  • 1
    Yes, it does. Running docker start jenkins works fine, and I can see the container when I run docker ps -a before I start it. – wheeler Dec 4 '17 at 12:51
  • It would be better to use docker-compose or the like as stated below. – Lucas Ramage Jun 25 at 12:23
2

Is there a reason why you try to do this with a systemd startup script and not with the docker mechanisms like restart: always?

We are using docker-compose.yml and it looks like this:

docker-compose.yml

version: '2'

services:
  jenkins:
    image: ###OWN_JENKINS_IMAGE###
    restart: always
    mem_limit: 2100m
    networks:
      - web
      - dockerhub
    environment:
      - "REVERSE_PROXY_PORT=8080"
      - "REVERSE_PROXY_HOSTNAME=####INTERNAL_HOSTNAME####"
      - "X_PROXY_PORT=443"
      - "X_PROXY_SCHEME=https"
      - "X_PROXY_NAME=####INTERNAL_HOSTNAME####"
    volumes:
      - ./data/jenkins:/var/jenkins_home
      - ./data/ssh/:/var/ssh/
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
      - ./data/volumes/lib64/libdevmapper.so.1.02:/usr/lib/libdevmapper.so.1.02
      - ./data/volumes/lib64/libdw.so.1:/usr/lib/libdw.so.1
      - ./data/volumes/lib64/libgcrypt.so.11:/usr/lib/libgcrypt.so.11
      - ./data/volumes/lib64/libsystemd-id128.so.0:/usr/lib/libsystemd-id128.so.0
      - ./data/volumes/lib64/libsystemd-journal.so.0:/usr/lib/libsystemd-journal.so.0

networks:
  web:
    external:
      name: web
  dockerhub:
    external:
      name: ####INTERNAL_HOSTNAME####

After starting the image with docker-compose up -d it will be restarted when there is a problem and as soon as the whole server below restarts.

  • 2
    Having Docker itself start the container is the way to go. – Michael Hampton Dec 4 '17 at 3:18
0

I agree with using Docker to start the container, and using the restart: always option.

As for the script & logging:

  • Are you overriding any defaults for the docker daemon? You might want to consider using an override file with just the non-default values rather than editing the systemd config files themselves.

  • Your output from systemctl: have you tried setting StandardOutput=tty to see what is happening as it happens?

  • Other logs: do you see any errors in other logs on your system such as /var/log/syslog or /var/log/messages? For systemd the default output for stderr is syslog.

I'm asking this because in my Unix/Linux experience you need to view the logs when system actions do not add up.

For more information on using Docker with systemd, you can check out this wiki.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.