Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am using logstash to collect logs from a group of webapps and send them to graylog2 for centralized viewing.

I have the following filter for tokenizing:

 grok {
   type => "webapps"
   pattern => "^%{TIME:timestamp} \[%{NOTSPACE:thread}\]%{SPACE}%{WORD:loglevel}%{SPACE}%{JAVACLASS:class} - %{GREEDYDATA:short_message}"
   drop => false
   debug => "true"
   add_tag => [ "%{loglevel}" ]
 }

Loglevel will be along the lines of TRACE, DEBUG, INFO, NOTICE, ERROR, FATAL.

Initially, everything was showing up as "Alert" (which has a numeric value of 5 in ruby/graylog-server and is marked in the sources as "unknown").

I then added a series of mutate filters, such as this ones:

 mutate {
   type => "webapps"
   tags => "INFO"
   add_tag => [ "ll_%{@level}", "mutated" ]
   replace => [ "@level", "6" ] # informational
 }

This got me closer to the proper loglevel/severity showing up in the webinterface, but all the "INFO" messages were showing up as being Debug messages.

I wrote a script to look through the elasticsearch data and set the level field based on what it should be.

  • Debug : 7
  • Informational : 6
  • Warning : 4
  • Error : 3
  • Critical : 2

with levels 0, 1 and 5 not being used, as the sources indicated these were reserved.

But this script is expensive in terms of resources, and I don't think it would work too well when the dataset it is trying to update is growing at a rate of hundreds or thousands of messages per second.

The other thing I noticed is that when I use something like "@source" in a tag, the proper value shows up. When I do "@level", I get that inserted as a literal, which indicates that it isn't a predefined field has had been indicated.

I've looked at the sources a little, but not exhaustively, so it's possible I've just missed it.

The question is, what do I need to change in my mutate filter to make "INFO" show up as "Informational" in the graylog2 interface?

share|improve this question

1 Answer 1

up vote 0 down vote accepted

Nothing I tried worked, and the mailing list wasn't able to help either.

What I wound up doing was to stop using logstash for this, and add in a logback-gelf jar and add to my logback configuration.

It has worked like a charm since then.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.