Nagios Notification Escalation

Question

I've got Nagios configured properly, and I'm looking to do some (minorly) strange things with notifications.

What we want to do is have the first three notifications fire off 5 minutes apart, then if nobody responds, the next 3 at 30 minutes apart, and beyond that, 60 minutes apart. I've got all this running properly as well.

The last part of what I want to do is where the problem occurs. I want these repeated notifications to NOT send overnight (between 23:00 and 08:00), but for all NEW notifications, those SHOULD send. What's happening is that for the third escalation (the 60-minute one) works fine up until 23:00, and then it reverts to the 5-minute level because it's outside the escalation_period I've set.

I thought I could get around it by setting another escalation period that runs during the 'nighttime' hours, but that didn't work either. Here's what I've got for the config:

define service{
        use                             generic-service
        host_name                       mercury
        service_description             ROB_TEST2
        check_command                   check_pop
        contact_groups                  robonly
        }

define serviceescalation{
        host_name                       mercury
        service_description             ROB_TEST2
        first_notification              3
        last_notification               5
        notification_interval           30
        contact_groups                  robonly
        }
define serviceescalation{
        host_name                       mercury
        service_description             ROB_TEST2
        first_notification              6
        last_notification               9999
        notification_interval           60
        contact_groups                  robonly
        escalation_period               daytime
        }
define serviceescalation{
        host_name                       mercury
        service_description             ROB_TEST2
        first_notification              6
        last_notification               9999
        notification_interval           60
        contact_groups                  nobody
        escalation_period               nighttime
        }

Any thoughts?

Answer 1

I think it would work if you specified the time period on the contact. Define the contact twice: once with notifications at night, and again with notifications only during the day.

define service{
    use                             generic-service
    host_name                       mercury
    service_description             ROB_TEST2
    check_command                   check_pop
    contacts                        rob_daytime, rob_nighttime
    }

define serviceescalation{
    host_name                       mercury
    service_description             ROB_TEST2
    first_notification              3
    last_notification               5
    notification_interval           30
    contacts                        rob_daytime, rob_nighttime
    }

define serviceescalation{
    host_name                       mercury
    service_description             ROB_TEST2
    first_notification              6
    last_notification               9999
    notification_interval           60
    contacts                        rob_daytime
    }

define contact{
    contact_name                    rob_daytime
    service_notification_period daytime
    ...
    }

define contact{
    contact_name                    rob_nighttime
    service_notification_period nighttime
    ...
    }

This should give you a good night's sleep even though an escalation has been running for a few days.

Note: I haven't tested this myself ;-)

Answer 2

Be very careful should you decide to limit anything by last_notification. At least with the versions I know, going into escalation does not reset that counter, which is relatively harmless. Transitions from WARN to CRIT also will not - which tends to end up deadly.