How can I parse an nginx msec field in logstash?

Question

I'm trying to handle an nginx access log in logstash.

In order to get millisecond accuracy for my timestamps, I'm using the $msec variable. This means that timestamps will be of the form 1430832725.814, where the integer part is a UNIX timestamp, and the fraction part is in milliseconds. Unfortunately, I can't find a logstash date format which can parse UNIX timestamps.

Is there another way to get nginx to log with millisecond accuracy? Or a way to get the logstash date filter to parse UNIX timestamps?

Answer 1

The UNIX format literal can handle fractional seconds, so it will be able to parse an $msec field.

Answer 2

firstly, using grok filter to map the $msec field to a output field

grok {
 match => { "message" =>\[%{GREEDYDATA:unix_timestamp}\] 
}

then using another Date filter to convert it to @timestamp field

  date {
     match => ["unix_timestamp", "UNIX"]
  }

If you want to map to fields other than @timestamp, refer to Logstash reference