1

I'm trying to handle an nginx access log in logstash.

In order to get millisecond accuracy for my timestamps, I'm using the $msec variable. This means that timestamps will be of the form 1430832725.814, where the integer part is a UNIX timestamp, and the fraction part is in milliseconds. Unfortunately, I can't find a logstash date format which can parse UNIX timestamps.

Is there another way to get nginx to log with millisecond accuracy? Or a way to get the logstash date filter to parse UNIX timestamps?

  • Have you read docs? «There are a few special exceptions.... “UNIX_MS” - will parse unix time in milliseconds since epoch» – Alexey Ten May 5 '15 at 13:53
  • I missed that. But I don't have milliseconds since the epoch. I have seconds, with millisecond resolution. – leedm777 May 5 '15 at 17:49
0

The UNIX format literal can handle fractional seconds, so it will be able to parse an $msec field.

0

firstly, using grok filter to map the $msec field to a output field

grok {
 match => { "message" =>\[%{GREEDYDATA:unix_timestamp}\] 
}

then using another Date filter to convert it to @timestamp field

  date {
     match => ["unix_timestamp", "UNIX"]
  }

If you want to map to fields other than @timestamp, refer to Logstash reference

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.