1

I'd like to send Apache HTTPd log statistics, such as 200 status counts to Graphite/Carbon. Logstash looks ideal but all the examples I've seen use Statsd to act as the state counter. This means spinning up a Statsd server (or enabling Statsd in Collectd 5.x).

Is there a way for Logstash to write counters directly to Graphite/carbon?

1

Yes, using the "metric" filter in logstash. In the default configuration, it will emit metric events every 5 seconds for a given field. By resetting the counter every 5 seconds, you can send the data straight to Graphite's carbon server for storing.

input {
    file {
        path => "/var/log/apache2/access.log"
    }
}

filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}" }
    }
    date {
        match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
    }

    # make sure response code is valid
    if [response] =~ /\d\d\d/ {
        metrics {
            # A counter field
            meter => "apache.response.%{host}.%{response}"
            add_tag => "metric"
            clear_interval => "5"
            flush_interval => "5"
        }
    }
}

output {
    #stdout { codec => rubydebug }

    graphite {
        fields_are_metrics => true
        # only send metrics collected in the filter
        include_metrics => ["^apache\.response\..*"]
        #host => "localhost"
        #port => "2003"
    }
}

Every 5 seconds, the following event is created:

{
    "@version" => "1",
    "@timestamp" => "2015-05-26T11:38:15.510Z",
    "message" => "ip-10-0-0-148",
    "apache.response.ip-10-0-0-145.401.count" => 1,
    "apache.response.ip-10-0-0-145.401.rate_1m" => 0.0,
    "apache.response.ip-10-0-0-145.401.rate_5m" => 0.0,
    "apache.response.ip-10-0-0-145.401.rate_15m" => 0.0,
    "tags" => [
        [0] "metric"
    ]
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.