2

I have the following sensu check defined in puppet

sensu::check { 'check_kubernetes_kubelet':
   interval    => 60,
   timeout     => 30,
   command     => 'check_http -H :::name::: -p 10248 -u /healthz -e "200 OK"',
   subscribers => ['kubernetes_minion'],
   contacts    => ['sensu_high'],
}

I want to add command retry to this check. This check should start alerting to its contacts only after N consecutive command executions return error.

In this example, say N is 2. Say a failure happens right after check_X. Then 60 seconds later check_X+1 will return error. After another 60 seconds check_X+2 will also fail. I want the alerting to start after check_X+2's error.


I think sensu filters may be useful. It looks like I need to create a filter like here. But then how do I associate that filter with this check? Do I need to define a custom handler too ?

2 Answers 2

0

Occurences is what you need to use.
occurrences => 2,

1
  • 1
    Yes we end up using occurrences too. But we needed to put the occurrences field in the custom field. So it looked like custom => { 'occurrences' => 3, },
    – Hakan Baba
    Commented May 3, 2018 at 7:53
1

You have to use handlers. When your check triggers and event, this gets to the handler or handlers you have defined for it (search for "handler" in checks). Then, each handler can have one or more filters associated.

From the doc:

When the Sensu server is processing an Event, it will check for the definition of a handler (or handlers). Prior to executing each Handler, the Sensu server will first apply any configured filter (or filters) for the Handler

2
  • Looking at the handler docs, what type of handler would you use? How would the filter for that look like? Looking at the enterprise edition, it looks like there is a handle_when parameter to sensu::check. However I do not see it in the open source sensu::check definition
    – Hakan Baba
    Commented Oct 26, 2017 at 5:29
  • The handler type depends on what you want to do with the data generated by the event. The two most used are "pipe", if you want to send it to a command (built-in or created by you), or "tcp/udp" to send the info to a socket. I'm not familiar with the other types... Regarding the filter, I think you need to use "occurences" sensuapp.org/docs/latest/reference/… Commented Oct 26, 2017 at 20:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .