69

I'm having trouble figuring out how to remove systemd units that no longer have files. They still seem to linger in the system somehow.

The old broken units I am trying to remove:

core@ip-172-16-32-83 ~ $ systemctl list-units --all firehose-router*
  UNIT                       LOAD      ACTIVE SUB    DESCRIPTION
<E2><97><8F> [email protected] not-found failed failed [email protected]
<E2><97><8F> [email protected] not-found failed failed [email protected]

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

2 loaded units listed.
To show all installed unit files use 'systemctl list-unit-files'.

The files do not exist, yet a reload still has these units lingering:

core@ip-172-16-32-83 ~ $ systemctl list-unit-files [email protected]
core@ip-172-16-32-83 ~ $ sudo systemctl daemon-reload
core@ip-172-16-32-83 ~ $ systemctl list-units --all firehose-router*
  UNIT                       LOAD      ACTIVE SUB    DESCRIPTION
<E2><97><8F> [email protected] not-found failed failed [email protected]
<E2><97><8F> [email protected] not-found failed failed [email protected]

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

2 loaded units listed.
To show all installed unit files use 'systemctl list-unit-files'.

There are no files related to them that I can find:

core@ip-172-16-32-83 ~ $ sudo find /var/run/systemd -name "*firehose-router*"
core@ip-172-16-32-83 ~ $ find /etc/systemd/ -name "*firehose-router*"
core@ip-172-16-32-83 ~ $ find /usr/lib/systemd/ -name "*firehose-router*"
core@ip-172-16-32-83 ~ $

So how do I get rid of these?

5
  • You don't mention it, but I guess it fails if you try to systemctl disable them, right?
    – dawud
    Jun 19, 2014 at 18:27
  • 2
    It just exist with 0. There is nothing to disable (it is already missing / disabled).
    – Andy Shinn
    Jun 19, 2014 at 18:31
  • I tried all the solutions suggested here (and below so far) and rebooted and nothing worked. The units I'm trying to remove were once installed or attempted to install, and then later apt purged. So apparently apt purge does not remove the configuration that is given to systemd. It's still not clear to me if these "not-found" units cause any problems. Sep 30, 2016 at 5:05
  • I suspect I might be able to delete files in /var/lib/systemd/deb-systemd-helper-enabled/ but am not sure if I want to mess with this. I appear to have two known dead ends there: mariadb.service.dsh-also and mysql.service.dsh-also Sep 30, 2016 at 5:20
  • @Elipticalview Make a backup of these files, just in case, and afterwards remove these?
    – gxx
    Feb 24, 2017 at 10:09

4 Answers 4

130

The command you're after is systemctl reset-failed

7
  • 3
    Thank you! I don't know why that was not more obvious to me...
    – Andy Shinn
    Jun 19, 2014 at 18:40
  • 7
    This worked for finally cleaning up a "real" service, but I have an alias service that refuses to go away. I tried disable, daemon-reloadand reset-failed but the alias service comes up as not-found inactive dead always. I also searched the disk for anything that matches the service name with no results. Jul 1, 2015 at 18:16
  • 8
    I just ran into this and systemctl stop <service> worked for me.
    – mpontillo
    Aug 8, 2016 at 21:58
  • 1
    The same applies for timers - they need to be stopped first before reset-failed can clean them up.
    – rustyx
    Feb 25, 2018 at 19:31
  • Kind of worked for me but I had to mask then unmask first for one of them. Also, had to stop one then mask and unmask after which reset-failed finally got rid of all.
    – tgunr
    Jul 18, 2019 at 21:44
4

When systemd analyzes unit definition files, it takes note of any other related units called out in the file - whether those other units exist or not.

$ systemctl --state=not-found --all
> ( ...prints list of 'not-found' units )

$ grep -r "<missing-unit>" /usr/lib/systemd/system
> ( returns files with references to <missing-unit> )

When a unit shows up as "not-found", it's not necessarily an error - all we know is, a local unit definition claims to have some relationship with it. This relationship might not be one we care about. For example, it could be "Before:" some other unit, but we don't use that other unit.

4
  • failed - happens when a unit has entered a failed state and can be reset with the systemctl reset-failed command
  • not-found - happens when you've removed a unit but systemd still has a reference to it, like when a unit gets enabled and a symlink gets placed in /etc/systemd/system, this can be fixed by removing references to the unit in /etc/systemd/system/*.wants/ then running systemctl daemon-reload

For example, assume the following bash script:

#!/bin/bash
# script.sh
while true
do
    sleep 1
done

And three systemd units: example-foo.service, example-bar.service, and example-baz.service

$ sudo systemctl cat example-{foo,bar,baz}.service
# /etc/systemd/system/example-foo.service
[Service]
ExecStart=/home/vagrant/script.sh
[Install]
WantedBy=multi-user.target

# /etc/systemd/system/example-bar.service
[Service]
ExecStart=/home/vagrant/script.sh
[Install]
WantedBy=multi-user.target

# /etc/systemd/system/example-baz.service
[Service]
ExecStart=/home/vagrant/script.sh
[Install]
WantedBy=multi-user.target

Now, let's start and enable the units. Observe how symlinks get created.

$ sudo systemctl start example-{foo,bar,baz}.service
$ sudo systemctl enable example-{foo,bar,baz}.service
Created symlink from /etc/systemd/system/multi-user.target.wants/example-foo.service to /etc/systemd/system/example-foo.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/example-bar.service to /etc/systemd/system/example-bar.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/example-baz.service to /etc/systemd/system/example-baz.service.

Confirm there are actually six files for our three units.

$ find /etc/systemd/system -name 'example*.service'
/etc/systemd/system/multi-user.target.wants/example-bar.service
/etc/systemd/system/multi-user.target.wants/example-foo.service
/etc/systemd/system/multi-user.target.wants/example-baz.service
/etc/systemd/system/example-bar.service
/etc/systemd/system/example-foo.service
/etc/systemd/system/example-baz.service

Now, check the state of all three units, they're running.

$ systemctl list-units example*
UNIT                LOAD   ACTIVE SUB     DESCRIPTION
example-bar.service loaded active running example-bar.service
example-baz.service loaded active running example-baz.service
example-foo.service loaded active running example-foo.service

Now, simulate a failure by sending a SIGKILL to example-foo.service. Observe how the unit is in a failed state.

$ sudo systemctl kill -s KILL example-foo.service
$ systemctl list-units example*
  UNIT                LOAD   ACTIVE SUB     DESCRIPTION
  example-bar.service loaded active running example-bar.service
  example-baz.service loaded active running example-baz.service
● example-foo.service loaded failed failed  example-foo.service

To reset a unit in a failed state use the systemctl reset-failed command. Observe how the unit is now in an inactive state.

$ sudo systemctl reset-failed
$ systemctl list-units example*
UNIT                LOAD   ACTIVE SUB     DESCRIPTION
example-bar.service loaded active running example-bar.service
example-baz.service loaded active running example-baz.service

...
$ systemctl list-units --all example*
UNIT                LOAD   ACTIVE   SUB     DESCRIPTION
example-bar.service loaded active   running example-bar.service
example-baz.service loaded active   running example-baz.service
example-foo.service loaded inactive dead    example-foo.service

Okay, now let's remove the example-bar.service unit. Observe how the unit is in a not-found state; however, the example-bar.service broken symlink is still in /etc/systemd/system/multi-user.target.wants

$ sudo rm /etc/systemd/system/example-bar.service
$ sudo systemctl daemon-reload
$ sudo systemctl stop example-bar.service
Failed to stop example-bar.service: Unit example-bar.service not loaded.
$ systemctl list-units --all example*
  UNIT                LOAD      ACTIVE   SUB     DESCRIPTION
● example-bar.service not-found inactive dead    example-bar.service
  example-baz.service loaded    active   running example-baz.service
  example-foo.service loaded    inactive dead    example-foo.service
$ find /etc/systemd/system -name 'example*.service'
/etc/systemd/system/multi-user.target.wants/example-bar.service
/etc/systemd/system/multi-user.target.wants/example-foo.service
/etc/systemd/system/multi-user.target.wants/example-baz.service
/etc/systemd/system/example-foo.service
/etc/systemd/system/example-baz.service

Remove the broken symlink and confirm the example-bar.service unit is gone.

$ sudo rm /etc/systemd/system/multi-user.target.wants/example-bar.service
$ sudo systemctl daemon-reload
$ systemctl list-units --all example*
UNIT                LOAD   ACTIVE   SUB     DESCRIPTION
example-baz.service loaded active   running example-baz.service
example-foo.service loaded inactive dead    example-foo.service
0

It seems that systemd maintains links but does not know what to do with them when you delete the unit file.

You could try to remove them manually in /etc/systemd/system/suspend.target.wants/ and such but of course systemctl reset-failed from a previous answer sounds like a better option.

$ cd /etc/systemd/system
$ sudo mv lock.service /tmp 
$ sudo systemctl disable lock.service
Failed to disable unit: No such file or directory
$ sudo mv /tmp/lock.service .
$ sudo systemctl disable lock.service
Removed /etc/systemd/system/suspend.target.wants/lock.service.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .