Tagged Questions

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

learn more… | top users | synonyms

0
votes
0answers
4 views

logstash / kibana can't connect to instance

I don't remember changing anything at all on my logstash server. As of just yesterday it was working fine! And I used it to run some queries. However.. today when I went to my logstash page, I see ...
-1
votes
1answer
17 views

How to send a daily report [closed]

Is there a ready-made solution to send a daily monitoring report by email via Collectd, Logstash, ElasticSearch or Kibana as Logwatch does it but for multiple servers ? Many thanks in advance for ...
0
votes
0answers
33 views

kibana: no indices error

None of the existing answers helped, so here a new question. Usecase: Redirecting syslog (or) monitoring static file. I have successfully installed logstash (1.4.2), elasticsearch(1.1.1) and ...
0
votes
1answer
69 views

Elasticsearch River JDBC - Avoiding 'SELECT *'

I'd like to syncronize some MySQL tables to an ElasticSearch index, you know, for search. But almost every examples I can find show a big bad fat "SELECT * FROM", it's OK for an example, but I'm in ...
0
votes
0answers
19 views

ElasticSearch cluster with remote nodes

Introduction Right now, I have three ElasticSearch servers (in Germany, US and Middle East(ME)). Each one has the same indexes but different data (they collect logs from local servers). Goal I want ...
0
votes
0answers
26 views

Reduce Elasticsearch log level

I have been trying to reduce the number of messages being logged by elastic search, however no matter what levels i put on each value (WARN, ERROR) i always see alot of DEBUG/WARN messages... I want ...
0
votes
1answer
173 views

Elasticsearch is using way too much disk space

I have a CentOS 6.5 server on which I installed Elasticsearch 1.3.2. My elasticsearch.yml configuration file is a minimal modification of the one shipping with elasticsearch as a default. Once ...
0
votes
0answers
45 views

Logstash/collectd buffer to files on network fault

I have read this: http://logstash.net/docs/1.4.2/life-of-an-event An example setup: collectd ( on local machine ) ----[via UDP]---> logstash (on local machine) ---[via elasticsearch output ...
0
votes
0answers
57 views

Kopf can not access elasticsearch

I have a setup with ElasticSearch, Logstash and Kibana working. With Kibana, I can access the data of the ElasticSearch from a remote host. I installed the latest kopf plugin with : ...
0
votes
1answer
2k views

Configure Nginx, Kibana, Elasticsearch

My setup Nginx + Kibana - same box different domain, sub-domain (nginx - example.com, kibana - kibana.example.com) Elasticsearch - 192.168.100.31 I have used the config defined below. Replaced ...
-2
votes
1answer
107 views

How do I know how the hacker gained access to my machine? [duplicate]

I own a dedicated machine and today I got an email from my host saying that my server will be blocked if I don't stop abusing the network. I am not a spammer or hacker. Someone was using my machine to ...
2
votes
1answer
57 views

ES fills up heap slowly over time and hangs at 14GB, while biggest index is 164MB?

I have an issue with Elasticsearch, at some times, it tries to run GC continously since this one is unable to free because heap size set to 14GB (min and max) is said to be completely allocated: ...
0
votes
0answers
25 views

ElasticSearch mappings for Apache URLs

So I'm feeding Apache access logs into my ElasticSearch cluster, but those darned developers keep crashing my cluster with their ad-hoc queries. Can anyone share what they're using to facilitate ...
0
votes
1answer
497 views

Installing Graylog2 with puppet

I have manged to install Graylog2 from scratch by downloading all the DEB files and making the configurations. Now I try to install Graylog2 with the help of puppet in an environment and had no luck ...
0
votes
0answers
155 views

How can I add basic auth to an apache proxy while using CORS headers?

Background I use grafana for displaying graphs of our server-metrics. Grafana is a JS-app which in our case is getting the data from graphite and storing search-queries in elasticsearch. All three ...
1
vote
0answers
179 views

ElasticSearch major slowdown upon big aggregations

My goal is to use ElasticSearch v1.3.2 for analyzing product cross-sales, so I need to filter for the receipts of interest (having an alcoholic product for example) and finding top selling products of ...
1
vote
0answers
208 views

JVM process resident set size “equals” max heap size, not current heap size

After a few reading about jvm memory (here, here, here, others I forgot...), I am expecting the resident set size of my java process to be roughly equal to the current heap space capacity. That's not ...
0
votes
1answer
253 views

Could not access pid file for elasticsearch

I am not a die-hard linux engineer, but I am having some problems with starting Elastic Search. Some stats: Server: Ubuntu Server 11.04 Elastic Search: 1.2.3 (installed with appitude) When I start ...
0
votes
1answer
45 views

Monitoring solution that doesn't duplicate source data?

I need to analyze data stored in multiple databases. Each of them contains a few TB of logs and sensor data. If I use Splunk or ElasticSearch/Kibana, I see 2 solutions: Batch import everything ...
1
vote
1answer
413 views

Graylog2 server cannot connect to Elasticsearch

I try to follow Digital Ocean manual except using last versions of components. My elasticsearch.yml: cluster.name: graylog2 network.bind_host: localhost network.publish_host: localhost ...
1
vote
1answer
228 views

Elasticsearch Cluster State immediately goes Yellow after first index created

I run a number of standalone Logstash servers to allow review of log files from web application servers. One of these recently reported a Yellow cluster state due to unassigned shards. This is a ...
0
votes
1answer
270 views

How do I make a graceful shutdown (with zero replicas)?

I am running a three node (elasticsearch) cluster currently with index.number_of_replicas: 0. I'd like to bring one of the node down for maintenance. Do I have any options to do this without ...
0
votes
0answers
101 views

ElasticSearch upgrade from 0.90.5 to 1.2.2

we're planning an Elasticsearch cluster upgrade from 0.90.5 to 1.2.2 . As this will be my first "full cluster restart" upgrade just wanted to get some information that a procedure I've put in place is ...
2
votes
3answers
485 views

sudden peak in cpu usage

We are running a 4 node/machine elastic search cluster on 12 core, 96gb RAM, 4 spinning disk machines. under normal operation most cpu usage is user and around 5-10%. Every few days, one of the ...
0
votes
0answers
90 views

Is it possible to do autoscaling for elasticsearch in AWS at peak load?

I'm new to ElasticSearch but I'm already using ElasticSearch on my production site. I installed ElasticSearch in one EC2 instance. After using ElasticSearch for several month, now currently the CPU ...
0
votes
1answer
321 views

Numerical range in Kibana 3 not working

My tomcat access logs have last field as "%D - Time taken to process the request, in millis". I am trying to filter the logs which took more than some number of milliseconds, but it is showing all the ...
0
votes
0answers
86 views

indexing a ~3gb database into elasticsearch: how?

I have a new database of 2 million records (around 3gb when dumped json) produced every several days. I want to quickly put it into ElasticSearch. That's what I do right now: Create a new index and ...
1
vote
1answer
500 views

Using Logstash as shipper?

We are shipping logs from servers and using THE logstash on each server for shipping. So we read logs from the glob "/root/Desktop/Logstash-Input/**/*_log". input { file{ ...
0
votes
1answer
1k views

Problems configuring logstash for email output

I'm trying to configure logstash to send email alerts and log output in elasticsearch / kibana. I have the logs successfully syncing via rsyslog, but I get the following error when I run ...
0
votes
1answer
459 views

Iptables config breaks Java + Elastic Search communication

I am trying to set up a firewall for a server hosting a java app and ES. Both are on the same server and communicate to each other. The problem I am having is that my firewall configuration prevents ...
1
vote
1answer
2k views

Unable to configure Nginx as a reverse proxy for elasticsearch on another server

I'm trying to use Kibana on a server A to access elasticsearch on a server B. elasticsearch is protected with a basic auth authentication on server B. Kibana is also protected with basic ...
0
votes
1answer
270 views

iptables to allow input and output traffic to and from web server only

I have an Elastic Search server which seems to have been exploited (it's being used for a DDoS attack having had NO firewall for about a month). As a temporary measure while I create a new one I was ...
0
votes
2answers
156 views

What are records in 'netstat' command are when using logstash?

I use logstash with embedded elasticsearch. My logstash config looks like the following: output { tdout { debug => true debug_format => "json"} elasticsearch { embedded => true ...
1
vote
2answers
182 views

PHP - does curl run asynchroniously on my system? [closed]

I'm building a web app. I have a database of books indexed in ElasticSearch and REST API written in PHP. In the app, there's a search box, where I type a name of the book and JS script calls the ...
1
vote
1answer
3k views

elasticsearch with kibana displays empty dashboard

I have the following is showing up when I pull up my kibana dashboard: http://192.168.10.25/#/dashboard {{dashboard.current.title}} When I tail my logs I see the following 2014/05/14 13:31:45 ...
0
votes
0answers
232 views

rsyslog - How to flush queue?

I'm using rsyslog to get logs from haproxy and put them into logstash for elasticsearch/kibana. Everything was working ok, but i've found something strange in rsyslog. I've found that i've got ...
3
votes
1answer
718 views

How best to monitor logstash?

I've seen this question on the mailing list a few times but haven't had a satisfactory answer. How best to monitor that the pipeline isn't stuck? Clients -> logstash -> elasticsearch. Logstash and ...
0
votes
2answers
360 views

Can I run elasticsearch on a single server?

I use elasticsearch as part of a Logstash stack, in which all of the components of the stack are installed on the same server. The purpose of this is to expose application logs to developers for ...
0
votes
0answers
76 views

What should I use to monitor elasticsearch health?

I am planning on using elasticsearch in production for the first time. What statistics is it recommended that I monitor? I am currently monitoring: /_nodes/stats for jvm.mem.heap_used_percent over ...
1
vote
1answer
149 views

Elasticsearch on EC2, webserver on Heroku // EC2 ports only accessible from Heroku

I have an instance of Elasticsearch facing the internet on an EC2. On the other hand I have my webservice as a Heroku app. I want to secure the Elasticsearch instance and allow inbound traffic only ...
1
vote
1answer
4k views

Nothing appearing in kibana dashboard

I have installed Logstash ElasticSearch Kibana on an EC2 instance. I can visit http://example.com:9200 Here I get { "status" : 200, "name" : "Aleta Ogord", "version" : { "number" : ...
0
votes
1answer
612 views

Connection hangs & never connects between ec2 instances over port 9200; fast from outside AWS

I am trying to connect from a web server EC2 instance to an ElasticSearch server ec2 instance. The connection is slow to nonexistent when connecting from the EC2, but very fast when connecting from a ...
3
votes
1answer
2k views

ElasticSearch Server Randomly Stops Working

I have 2 ES servers that are being fed by 1 logstash server and viewing the logs in Kibana. This is a POC to work out any issues before going into production. The system has ran for ~1 month and every ...
-2
votes
1answer
372 views

Elasticsearch throws Could not reserve enough space for object heap

It seems that Elasticsearch stops working this morning. When i launch : sudo service elasticsearch start, it says : Starting ElasticSearch Server Done But when i check the status, it gives me : ...
0
votes
1answer
261 views

Elasticsearch stops working suddenly

I have deployed Elasticsearch on a development ubunto VPS. We use that VPS only to host Api for mobile apps. Everything worked good till today, elasticsearch stops working. I tried to restart it, it ...
0
votes
1answer
640 views

ElasticSearch Multiple Indexes and Routing

Currently I have the following setup. Syslog --> Logstash --> ElasticSearch --> Kibana Logstash is creating a daily index ...
0
votes
0answers
484 views

Output Logstach to Elasticsearch cluster

I have multiple EC2 instances running Logstash and Elasticsearch. I'd like to output everything into ES cluster. output { stdout { debug => true debug_format => "json" } ...
0
votes
1answer
536 views

Multiple instances of Logstash + Elasticsearch on AWS

I have setup a Cloudformation template to start an instance where Logstash and Elasticsearch (not embedded) are installed. I have standard tcp input configured on Logstash and output it to an ...
0
votes
0answers
175 views

graylog elasticsearch install cannot change open file limit on reboot

I have a problem with our graylog 2 install. The elasticsearch process keeps on running out of file handles, this is in the //opt/elasticsearch/logs/graylog2.log log file. [2014-01-09 ...
3
votes
1answer
2k views

Logstash Server Log FIle Flooded With “Failed to flush outgoing items” Error Messages

I'm new to Logstash, trying to get it running following the tutorial in "The Logstash Book". On page 44, the guide suggests to tail the Logstash process' main log file called central.log. After ...