2

I have a Centos 7 Box on which I have installed Nagios and then migrate all the config files from an old Centos box.

Everything looks fine, nagios -v doesn't return any error.

However, systemd is unable to start the service and give me a timeout. Find below the result of systemctl -l status nagios.service:

● nagios.service - Nagios Network Monitoring
   Loaded: loaded (/usr/lib/systemd/system/nagios.service; disabled; vendor preset: disabled)
   Active: failed (Result: timeout) since Fri 2016-02-05 10:52:55 CET; 13min ago
     Docs: https://www.nagios.org/documentation/
  Process: 2259 ExecStart=/usr/sbin/nagios -d /etc/nagios/nagios.cfg (code=exited, status=0/SUCCESS)
  Process: 2257 ExecStartPre=/usr/sbin/nagios -v /etc/nagios/nagios.cfg (code=exited, status=0/SUCCESS)

Feb 05 10:52:52 nagios.adflux.net nagios[2261]: SERVICE ALERT: VM-CRO-JIRA2;Drive Space C:;CRITICAL;SOFT;1;CRITICAL - Socket timeout after 10 seconds
Feb 05 10:52:52 nagios.adflux.net nagios[2261]: SERVICE ALERT: ESXi-ls1;PING;WARNING;SOFT;1;PING WARNING - Packet loss = 33%, RTA = 80.47 ms
Feb 05 10:52:55 nagios.adflux.net systemd[1]: nagios.service start operation timed out. Terminating.
Feb 05 10:52:55 nagios.adflux.net nagios[2261]: Caught SIGTERM, shutting down...
Feb 05 10:52:55 nagios.adflux.net nagios[2268]: Caught SIGTERM, shutting down...
Feb 05 10:52:55 nagios.adflux.net nagios[2261]: Successfully shutdown... (PID=2261)
Feb 05 10:52:55 nagios.adflux.net nagios[2261]: Event broker module 'NERD' deinitialized successfully.
Feb 05 10:52:55 nagios.adflux.net systemd[1]: Failed to start Nagios Network Monitoring.
Feb 05 10:52:55 nagios.adflux.net systemd[1]: Unit nagios.service entered failed state.
Feb 05 10:52:55 nagios.adflux.net systemd[1]: nagios.service failed.

No further error found on the logs (or at least, where I looked, maybe.... most probably I'm missing something here).

Running the command /sbin/nagios /etc/nagios/nagios.cfgstart the monitoring service and everything run as expected. But this doesn't solve my issue since Nagios isn't started as a daemon here and is link to my shell. This indicate me that the issue is not caused by Nagios but by systemd itself.

Any clue on that will be appreciate.

Many thanks.

Improve this question
2
  • Can you show us the contents of /usr/lib/systemd/system/nagios.service ?
    – lazyfrosch
    Feb 8, 2016 at 6:35
  • Sure thing. Here you go: [root@nagios ~]# cat /usr/lib/systemd/system/nagios.service [Unit] Description=Nagios Network Monitoring After=network.target Documentation=https://www.nagios.org/documentation/ [Service] Type=forking User=nagios Group=nagios PIDFile=/var/run/nagios/nagios.pid # Verify Nagios config before start as upstream suggested ExecStartPre=/usr/sbin/nagios -v /etc/nagios/nagios.cfg ExecStart=/usr/sbin/nagios -d /etc/nagios/nagios.cfg ExecStopPost=/usr/bin/rm -f /var/spool/nagios/cmd/nagios.cmd [Install] WantedBy=multi-user.target
    – Benkkei
    Feb 8, 2016 at 7:56

4 Answers 4

Reset to default
2

It seems like Nagios is not properly forking into background -d option, what systemd expects here due to type=forking.

So systemd counts a non-fork as a timeout during start. That might be related due to NERD, or another problem.

You could run Nagios in foreground by:

cp /usr/lib/systemd/system/nagios.service /etc/systemd/system/nagios.service
vim /etc/systemd/system/nagios.service
# remove Type=forking and -d in cmd line of nagios
systemctl daemon-reload
systemctl restart nagios.service

Nevertheless, there is a bug...

Improve this answer
2
  • Hello, Thanks for your reply. however, I'm not sure to fully understand it. (Sorry, I'm not use to systemD yet). I don't see the purpose of this command: cp /usr/lib/systemd/system/nagios.service /etc/systemd/system/nagios.service for instance and therefore I'm not sure which nagios.services file you want me to edit.
    – Benkkei
    Feb 11, 2016 at 10:44
  • I've updated the code a bit, the service unit in /etc overrides the one in /lib
    – lazyfrosch
    Feb 11, 2016 at 17:23
1

You've updated the Nagios installation far beyond the old configuration options. Despite -V showing all flight is fine.

I had the same issue because I didnt include any of the diffs in the nagios.cfg.rpmnew file.

I reinstalled after backing up my config and then added my changes to the new configuration.

Improve this answer
1
  • +1 on diffs with rpmnew files. The config changed a lot. Using the new cfg file fixed this for me.
    – Csongor Fagyal
    May 31, 2016 at 11:54
0

You have already solved your problem by overriding the default SystemD configuration. I had a similar problem, my Nagios in a Centos 7 machine stopped working after a recent automatic upgrade. The problem was that my /etc/nagios/nagios.cfg configuration defined a PID different than the expected by the SystemD configuration (/var/run/nagios.pid vs /var/run/nagios/nagios.pid) and so SystemD could not detect that the Nagios daemon actually started. The messages I got were similar to this:

systemctl status nagios.service -l
 nagios.service - Nagios Network Monitoring
   Loaded: loaded (/usr/lib/systemd/system/nagios.service; disabled; vendor preset: disabled)
   Active: failed (Result: timeout) since jue 2016-02-18 12:33:05 UTC; 1min 43s ago
     Docs: https://www.nagios.org/documentation/
  Process: 26986 ExecStart=/usr/sbin/nagios -d /etc/nagios/nagios.cfg (code=exited, status=0/SUCCESS)
  Process: 26985 ExecStartPre=/usr/sbin/nagios -v /etc/nagios/nagios.cfg (code=exited, status=0/SUCCESS)

feb 18 12:33:04 mvdatos.com nagios[26989]: wproc: 'Core Worker 26992' seems to be choked. ret = -1; bufsize = 124: errno = 32 (Broken pipe)
feb 18 12:33:04 mvdatos.com nagios[26989]: wproc: Socket to worker Core Worker 26991 broken, removing
feb 18 12:33:04 mvdatos.com nagios[26989]: wproc: Socket to worker Core Worker 26992 broken, removing
feb 18 12:33:04 mvdatos.com nagios[26989]: wproc: Socket to worker Core Worker 26993 broken, removing
feb 18 12:33:04 mvdatos.com nagios[26989]: Successfully shutdown... (PID=26989)
feb 18 12:33:05 mvdatos.com systemd[1]: Failed to start Nagios Network Monitoring.
feb 18 12:33:05 mvdatos.com systemd[1]: Unit nagios.service entered failed state.
feb 18 12:33:05 mvdatos.com systemd[1]: nagios.service failed.
feb 18 12:33:35 mvdatos.com systemd[1]: Stopped Nagios Network Monitoring.
feb 18 12:33:36 mvdatos.com systemd[1]: Stopped Nagios Network Monitoring.

Running the daemon directly (/usr/sbin/nagios -d /etc/nagios/nagios.cfg) worked perfectly, hinting the problem was something related to SystemD, not Nagios.

Hope this help somebody else with a SystemD timing out when starting Nagios.

Improve this answer
0

This answer assumes you are installing using yum from EPEL. If you install from source, the files may be in different locations but the ideas are the same. Make sure the nagios user has permission to these files.

Look for this section in /etc/nagios/nagios.cfg

# LOCK FILE
# This is the lockfile that Nagios will use to store its PID number
# in when it is running in daemon mode.
lock_file=/var/run/nagios/nagios.pid

That is the file Nagios will create when it starts up. Older versions might use nagios.lock, or the path might be something like /usr/local/nagios/

systemd specifies the pid file it is looking for in /lib/systemd/system/nagios.service

PIDFile=/var/run/nagios/nagios.pid

This needs to be the same as lock_file in your nagios.cfg, or systemd will timeout and stop your nagios service.

Since you have migrated configurations files and such over from an older Nagios install, make sure you change all your paths in the configuration files, ie nagios.cfg,objects.cfg,commands.cfg, etc . You will probably also have to move around your custom scripts inlibexec`, among other things.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .