logstash is a tool for collecting and distributing log events.

learn more… | top users | synonyms

-1
votes
1answer
28 views

Statistical information regarding scaling ElasticSearch [duplicate]

I have a set-up with several Logstash nodes sending input to ElasticSearch, and have a kibana server which allows me to visualize this. The current infrastructure is pretty simplistic and is on ...
0
votes
0answers
47 views

Kibana not finding local files e.g. require JS, CSS

I'm new to linux/ubuntu and the ELK stack and having an issue getting kibana to work. I've unpacked all three into this folder and started kibana and elastic search: cd /home/ubuntu ls ...
0
votes
0answers
168 views

logstash / kibana can't connect to instance

I don't remember changing anything at all on my logstash server. As of just yesterday it was working fine! And I used it to run some queries. However.. today when I went to my logstash page, I see ...
1
vote
1answer
47 views

Logstash gradually losing sockets

Using logstash 1.4.2 pushing to an elastic search 1.1.1 instance on a separate server. Pulling log items from AWS SQS. Everything works fine the first several hours or so, but then the following ...
0
votes
1answer
180 views

How to send Windows Performance counters to Logstash + Kibana?

I would like to setup monitoring for system resources on my Windows servers. I've noticed a common configuration in Linux is to use collectd daemon to get system metrics information. From collectd ...
0
votes
1answer
96 views

kibana: no indices error

None of the existing answers helped, so here a new question. Usecase: Redirecting syslog (or) monitoring static file. I have successfully installed logstash (1.4.2), elasticsearch(1.1.1) and ...
0
votes
2answers
268 views

Logstash Forwarder doesn't start up with chkconfig in CentOS 5

I have set up logstash-forwarder on a CentOS 5 machine installing it from this RPM: http://download.elasticsearch.org/logstash-forwarder/packages/logstash-forwarder-0.3.1-1.x86_64.rpm When I went ...
0
votes
0answers
17 views

Indexing individual cookies with Logstash

I'm working on some analysis of Apache logs. Our logs happen to capture the cookies the browser communicates it has and we log it with the request. Right now I can import that long string fine, ...
0
votes
1answer
33 views

install zeromq for logstash in Solaris 5.10

used this: http://zeromq.org/distro:debian to get zeromq in Ubuntu to use it with logstash. Want to achieve same within Solaris. Read that zeromq supports Solaris but only provides a tarball. How to ...
0
votes
1answer
99 views

Statistics of logstash memory consumption [duplicate]

I want to install it on a really small server and I want to see if it will be capable of running it with some dignity. So, does anyone have any memory and CPU consumption statistics of running ...
0
votes
0answers
89 views

Logstash/collectd buffer to files on network fault

I have read this: http://logstash.net/docs/1.4.2/life-of-an-event An example setup: collectd ( on local machine ) ----[via UDP]---> logstash (on local machine) ---[via elasticsearch output ...
1
vote
1answer
443 views

Logstash-forwarder is throwing SSL errors

I got this task handed over to by my colleage and this is the background. He got ELK (Elasticsearch, Logstash and Kibana) stack working with our RHEL 6.2 servers, by using the regular method of ...
0
votes
1answer
116 views

Logstash not reading logs unless world-readable

I recently set up Logstash with Kibana and Elasticsearch, but am having a problem whereby it's not reading logs unless the directory permissions are world read/execute (and the files at least world ...
0
votes
1answer
110 views

What software can I use to retrieve log files from multiple remote servers via SSH? [closed]

I know I can do this using scripts, but I'd like to know if there is such a thing as a "ready made" software which lets me configure the host (and file paths) which I'd like to fetch (at regular ...
0
votes
0answers
49 views

Logstash from RPM fails to run under nice command

I downloaded the Logstash RPM logtsash-1.4.2-1_2c0f5a1.noarch from the official site, but have been having problems getting it to run from the init script. I get no log data passed to ...
0
votes
1answer
125 views

RSyslog sends logs to logstash with host as IP not FQDN

I am having a issue. I am trying to send apache/nginx logs to logstash server. The problem is that rsyslog sends host as IP not the servers FQDN. Solutions that I have tried: PreserveFQDN on (did ...
1
vote
1answer
1k views

Logstash, Kibana and email alerts

I am trying to setup email alerts using logstash. Right now it emails me EVERY time the pattern "Error" is parsed into my log file which can lead to a lot of unnecessary emails. I'd like to create a ...
1
vote
1answer
141 views

How to include a server identifier in nxlog output and reference it in logstash

If I have nxlog running on multiple IIS servers (say web1, web2, web3, web4). How can I add an identifier to the nxlog output which is being sent to logstash. Then within logstash I want to create a ...
0
votes
0answers
158 views

logstash output to a webservice from Symfony

Good Morning, I would like to send logstash output to a web service I created with Symfony. So far my web service work, I used php to try to send a log. Quick example in PHP $client = new ...
1
vote
1answer
1k views

Logstash parsing xml document containing multiple log entries

I'm currently evaluating whether logstash and elasticsearch are useful for our use-case. What I have is a log file containing multiple entries which is of the form <root> <entry> ...
0
votes
1answer
532 views

parse old IIS logs with logstash

I have moved to logstash (combined with elastic search and Kibana) from Awsats. However I need to import/parse a few old IIS log files using logstash. The files has been moved to logstash server. ...
0
votes
1answer
472 views

Numerical range in Kibana 3 not working

My tomcat access logs have last field as "%D - Time taken to process the request, in millis". I am trying to filter the logs which took more than some number of milliseconds, but it is showing all the ...
0
votes
1answer
274 views

Inputting bulk logs in Logstash (stored in different folders) using the “file” plugin without specifying individual filters

So I have a question about the basic functionality of logstash. I have many different .L01 files which I have exported into log files. Each .L01 file represents a host and the export function dumps it ...
2
votes
1answer
685 views

Using Logstash as shipper?

We are shipping logs from servers and using THE logstash on each server for shipping. So we read logs from the glob "/root/Desktop/Logstash-Input/**/*_log". input { file{ ...
4
votes
3answers
3k views

Failed tls handshake. Does not contain any IP SANs

I'm trying to set up logstash forwarder, but I have issues with making a proper secure channel. Trying to configure this with two ubuntu (server 14.04) machines running in virtualbox. They are 100% ...
1
vote
1answer
675 views

How to parse audit.log using logstash

I want to use logstash to collect a log file, and the format of the file was like this: type=USER_START msg=audit(1404170401.294:157): user pid=29228 uid=0 auid=0 ses=7972 ...
1
vote
1answer
280 views

Correlating data within Kibana

I'm looking at aggregating data within the whole ELK stack (Elasticsearch/Logstash/Kibana) and I'm getting lots of good data from my windows systems. It's working very well, but when i get Windows ...
-1
votes
1answer
104 views

Alerting on threshold crossing?

We are using logstash + elasticsearch + Kibana to parse, store and view our logs respectively. Now, we want a notification/alert when a threshold for a particular log or threshold for a particular ...
0
votes
1answer
2k views

Problems configuring logstash for email output

I'm trying to configure logstash to send email alerts and log output in elasticsearch / kibana. I have the logs successfully syncing via rsyslog, but I get the following error when I run ...
1
vote
1answer
112 views

“Tail” a logstash server query

Assuming I have a logstash server chocked full of logs being loaded regularly, is there a reasonably elegant way that I can tail the results of a continuously executing query on the logstash server ...
3
votes
2answers
446 views

Postfix - searching emails (logstash, greylog or other solution)

We are currently having ~100 servers and all of them are using remote syslog, so we have aggregated all logs on one server. The most questioned problem from our support team is: Has an email from ...
0
votes
2answers
186 views

What are records in 'netstat' command are when using logstash?

I use logstash with embedded elasticsearch. My logstash config looks like the following: output { tdout { debug => true debug_format => "json"} elasticsearch { embedded => true ...
1
vote
1answer
200 views

puppet-logstash module produces a blank logstash.conf

I am probably confused on the proper usage. I have setup the configuration of my shipper using Hiera: classes: - os::repo - profiles::logstash::shipper logstash::configfile: 'input_file': ...
2
votes
1answer
324 views

Windows Forward Events Missing User Data and Description

I have Subscription Events set to forward windows server 2008's Terminal Services/LocalSessionManager/Operational logs to another window's server 2008's Forward Events section. The Subscription Event ...
2
votes
1answer
1k views

Sending logs to remote server with rsyslog

I'm trying to setup centralized logging with Logstash, Elasticsearch and Kibana and am having trouble getting logs to my log server. Logstash is listening on TCP 5000 and is successfully receiving ...
3
votes
1answer
792 views

How best to monitor logstash?

I've seen this question on the mailing list a few times but haven't had a satisfactory answer. How best to monitor that the pipeline isn't stuck? Clients -> logstash -> elasticsearch. Logstash and ...
0
votes
2answers
400 views

Can I run elasticsearch on a single server?

I use elasticsearch as part of a Logstash stack, in which all of the components of the stack are installed on the same server. The purpose of this is to expose application logs to developers for ...
1
vote
2answers
3k views

Configuring Logstash when installed as a service [closed]

I have installed logstash as a service using the logstash APT repository on Ubuntu 13.10. So now I can run: dpkg -s logstash And it outputs: Package: logstash Status: install ok installed ...
1
vote
2answers
5k views

Nothing appearing in kibana dashboard

I have installed Logstash ElasticSearch Kibana on an EC2 instance. I can visit http://example.com:9200 Here I get { "status" : 200, "name" : "Aleta Ogord", "version" : { "number" : ...
-1
votes
1answer
1k views

lumberjack shipper init script

I'm using lumberjack shipper for logstash. I'm in need of init.d script to start lumberjack on centos system. I have the script for debian based systems but can't get one on Centos to work. I have ...
0
votes
0answers
531 views

Output Logstach to Elasticsearch cluster

I have multiple EC2 instances running Logstash and Elasticsearch. I'd like to output everything into ES cluster. output { stdout { debug => true debug_format => "json" } ...
0
votes
1answer
611 views

Multiple instances of Logstash + Elasticsearch on AWS

I have setup a Cloudformation template to start an instance where Logstash and Elasticsearch (not embedded) are installed. I have standard tcp input configured on Logstash and output it to an ...
0
votes
2answers
625 views

Java process restarting prematurely under runit

I'm attempting to run Logstash under supervision with runit. My run script for Logstash is simply #!/bin/sh SSL_CERT_DIR=/etc/ssl/certs exec /usr/bin/java -jar /usr/local/bin/logstash.jar agent -f ...
0
votes
0answers
941 views

Configuring LogStash properly w/ nxlog

I'm looking for a good configuration that sends most System, Application and Security events to an output. I would have thought this configuration would work, but it seems that I get no logs from ...
2
votes
1answer
2k views

What is the significance of the @ prefix in logstash field names?

The following logstash configuration is used to accept Windows Event Logs as json over a TCP connection and then after some filtering forward the result to Elastic search (source: ...
3
votes
1answer
2k views

Logstash Server Log FIle Flooded With “Failed to flush outgoing items” Error Messages

I'm new to Logstash, trying to get it running following the tutorial in "The Logstash Book". On page 44, the guide suggests to tail the Logstash process' main log file called central.log. After ...
1
vote
0answers
295 views

how to send individual data fields from logstash to zabbix

so we want to monitor the Tomcat access times with zabbix. I looked at logstash, thinking that it could parse the log messages, and it can. I get parts of the log message as individual fields in the ...
0
votes
0answers
249 views

Message Structure Different Between Log Shipper — Logstash and nxlog

I have a Windows Server 2008 (ugh) system that I'm pulling messages from with Logstash and nxlog. With nxlog as the log shipper, messages as they appear in the log file are sent to the parser: ...
0
votes
1answer
505 views

Log Rotation Causing nxlog Failure

I'm using nxlog to watch two vCenter log files on a Windows Server 2008 system and I have a very simple config to ship messages to Logstash. Nxlog began failing yesterday after a file was rotated. ...
0
votes
1answer
678 views

nxlog parse_csv fails on odd character in IIS log file

I am parsing IIS logs. This error shows in my nxlog eventlog: 2013-12-24 18:40:20 ERROR if-else failed at line 50, character 351 in C:\Program Files (x86)\nxlog\conf\nxlog.conf. statement ...