0

I'm setting up LogStash on Windows and made a test launch of logstash.bat from the console to see if it process records from the log4net file. Here is what it reported but I don't see any records in the target ES although there are records in the log file:

C:>logstash.bat agent -f logstash.conf

Using JAVA_HOME=C:\Program Files\Java\jre1.8.0_102 retrieved from C:\ProgramData\Oracle\java\javapath\java.exe
io/console not supported; tty will not be manipulated
Settings: Default pipeline workers: 4
Pipeline main started

logstash.conf (host, uid and pwd are correct so it's not a connectivity issue):

 input { 
    file {
       path => "C:\LogStash\logs\logfile"
type => "log4net"
       codec => multiline {
                pattern => "^(DEBUG|WARN|ERROR|INFO|FATAL)"
                negate => true
                what => previous
            }
       }
 }
 filter {
   if [type] == "log4net" {
    grok {
       match => [ "message", "(?m)%{TIMESTAMP_ISO8601:sourceTimestamp} \[Worker #%{NUMBER:threadId}\] %{LOGLEVEL:level} %{GREEDYDATA:tempMessage}" ]
     }
     mutate {
         replace => [ "message" , "%{tempMessage}" ]
         remove_field => [ "tempMessage" ]
     }
   }
 }
 output {
   elasticsearch {
     hosts => ["http://XXXXX:9200"]
     user => "XXXXX"
     password => "XXXXX"
     index => "logstash-%{+YYYY.MM.dd}"
     template_overwrite => true
   }
 }

1 Answer 1

0

It turns out that LogStash requires special handling when working with "historical" log files i.e. the files that either it already looked at or that are older than 24 hours.

start_position => "beginning" - forces LS to look at the files from the beginning assuming that all .sincedb* have been deleted in your profile/HOME directory.

ignore_older => 86400 (default!) - forces LS to ignore any files that are older than 24 hours.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .