2

I've configured logstash (v1.5.0), with a simple syslog input, as follows:

input {
  syslog {
    type => syslog
    port => 5514
  }
}

filter {
  kv {}
}

output {
  elasticsearch {
    cluster => "logs"
    host => "0.0.0.0"
    protocol => "transport"
  }
}

However it seems to be failing on some of the cron logs. The following line fails to parse with a _grokparsefailure_sysloginput:

<77>Jul 22 22:01:01 ip-172-31-2-48 run-parts(/etc/cron.hourly)[2599 finished 0yum-hourly.cron

The final JSON output is:

{
  "_index": "logstash-2015.07.22",
  "_type": "syslog",
  "_id": "AU63yLrC118PBgBqQxRA",
  "_score": null,
  "_source": {
    "message": "<77>Jul 22 22:01:01 ip-172-31-2-48 run-parts(/etc/cron.hourly)[2599 finished 0yum-hourly.cron\n",
    "@version": "1",
    "@timestamp": "2015-07-22T22:01:01.569Z",
    "type": "syslog",
    "host": "172.31.2.48",
    "tags": [
      "_grokparsefailure_sysloginput"
    ],
    "priority": 0,
    "severity": 0,
    "facility": 0,
    "facility_label": "kernel",
    "severity_label": "Emergency"
  },
  "fields": {
    "@timestamp": [
      1437602461569
    ]
  },
  "sort": [
    1437602461569
  ]
}

Any pointers?

  • I don't think that's a standard Syslog line format. What's the Grok pattern you're using? – GregL Jul 23 '15 at 15:12
4

The syslog input use grok internally, your message is probably not following the syslog standard 100%.

The solution in this link worked for me: http://kartar.net/2014/09/when-logstash-and-syslog-go-wrong/

The key info from the link is:

Replace the existing syslog block in the Logstash configuration with:

input {
  tcp {
    port => 514
    type => syslog
  }
  udp {
    port => 514
    type => syslog
  }
}

Next, replace the parsing element of our syslog input plugin using a grok filter plugin.

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
    }
  }
}

You can edit the filter matching ("grok") syntax now, to match your desired format. It's also possible to support multiple different syntaxes with creative use of if, else if, and else.

  • 1
    Please don't just post the link in an answer, but rather copy the pertinent bits and include them in the answer. Links die and pages get deleted, so keeping it all here makes the answer useful should that happen. – GregL Jul 24 '15 at 11:45
  • 1
    @GregL: edited to add the config bits from the link. – willbradley Dec 11 '15 at 1:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.