0

I'm using following system/package:

$ cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core) 
$ rpm -q filebeat
filebeat-1.3.0-1.x86_64
$ 

with /etc/filebeat/filebeat.yml:

$ cat /etc/filebeat/filebeat.yml 
filebeat:
  prospectors:
    -
      paths:
        - /var/log/*.log
      input_type: log
  registry_file: /var/lib/filebeat/registry
output:
  elasticsearch:
    hosts: ["localhost:9200"]
shipper:
logging:
  to_syslog: true
  files:
$ 

kibana

"message": "Sep 8 10:20:01 X CROND[11586]: (root) CMD (/usr/lib64/sa/sa1 1 1)",

  • How can I use timestamp from message as @timestamp in Kibana?
  • How can I separate out rest of message (daemon, etc.) into separate fields?
  • Please bear with me, if I didn't ask my question properly, I'm only learning) – alexus Sep 8 '16 at 18:14
0

The message field is text, not something Kibana knows how to use as a timestamp. You need to add some additional parsing in order to convert the timestamp from your log file into a date data type. You can learn more about Elasticsearch data types by reading the relevant documentation.

You've configured Filebeat to output directly to Elasticsearch. In order to parse the timestamp (and potentially other fields) from your log file, you'll need to configure Filebeat to output to Logstash instead. Logstash can then be used to modify your log data, parsing the timestamp as well as doing other pattern matching.

Having said all that, it looks like you might be using Filebeat to read a syslog file. That's fine, but it's also possible for Logstash to directly receive syslog data, which could simplify your overall setup.

  • I think the OP's question could be generalized as follows: Let's say he had a time field defined above and it contained some time stamp value. He wants Kibana to use that field as the official "timestamp" rather than defaulting to Elasticsearch's automatically-generated timestamp field. – Howiecamp Jul 16 '17 at 2:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.