Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

learn more… | top users | synonyms (1)

0
votes
0answers
23 views

How to add Elasticsearch as a service in Ambari

I was wondering to know how can I add Elasticsearch as a service to Ambari portal and monitor it through that portal? ... I know it's doable via Ambari Stacks, but I don't know how doing it. I found ...
1
vote
0answers
19 views

Visualize multiline ruby exceptions in kibana 4

I have setup the latest version of Kibana4 ElasticSearch stack. The logs are being pooled from remote app sources which are running on Ruby. I want to search for Multi Line exceptions created by ...
1
vote
1answer
23 views

ElasticSearch container on CoreOS with persistent storage

I'm trying to run an ElasticSearch cluster on Docker and CoreOS [stable] on DigitalOcean. To get persistent storage working, I tried pretty much every option and running out of ideas. Pretty much ...
0
votes
0answers
17 views

ElasticSearch replica shards unassigned

I'm trying to set an ElasticSearch system with 2 nodes (for a failover system), but the replication doesn't work on the second node. More explanation : So first I create a first node and I create an ...
0
votes
0answers
41 views

Not able to access Kibana GUI with http://Ip:5601/

I have installed Elastisearch 2.1.0 and kibana 4.3.0 in single machine. Kibana.yml Configurations : # Kibana is served by a back end server. This controls which port to use. server.port: 5601 # The ...
0
votes
0answers
21 views

PG, Redis, and Elasticsearch replication for 2 distant metal servers

After years of running a web app on ~15 separate VPS, I'm switching to a single physical server*. App uses 3 DBs: Postgres, Redis, and Elasticsearch, and can update to latest version on all 3. ...
0
votes
2answers
46 views

Kibana: No Living connections

I have ES 1.5.2 cluster with Kibana 4.0.1. This consists of 4 data/master nodes along with 4 client nodes. When I'm trying to load kibana sometimes it takes longer time to load or sometimes I need to ...
1
vote
2answers
55 views

Nginx reverse proxy for elasticsearch - inserting filters

I've got an elasticsearch instance, with an Nginx reverse proxy sat in front of it, implementing a URL level access control mechanism - approximately as described in this article This is giving me ...
0
votes
1answer
55 views

ElasticSearch Delayed Indexing

I currently have the following setup: syslog-ng servers --> Logstash --> ElasticSearch The syslog-ng servers are load balanced and write to a SAN location where Logstash just tails the files and ...
0
votes
2answers
47 views

Logstash filtering syslog by host group?

I've got an Elasticsearch/Logstash/Kibana instance running, which I'm merrily stuffing with syslogs from a variety of hosts. Having built it to scale - with multiple logstash syslogd listeners, and ...
1
vote
0answers
45 views

Filebeat and downstream availability

I read here and there that a broker (like Redis) might not be required in the log pipeline (typically ELK) when Filebeat is used. From Filebeat's official page: [Filebeat] is intelligent enough ...
0
votes
1answer
209 views

ELK: LogStash to read log files from remote Samba-mapped network drives

I'm new to ELK, and I would like to set up a solution to index Microsoft IIS and applicative .NET logs with ES. I'm aware about different approaches: 1) [app servers: log files ➔ Logstash] ➔ ...
1
vote
0answers
18 views

elasticsearch creating files but not releasing them until I restart the service

I am running elastcsearch 0.90.5 and recently I have been having issues with the disk running out of space. This is not caused by my indexes increasing in size but by the elasticsearch service ...
0
votes
1answer
38 views

Elasticsearch Cluster Configuration

Brand new to Elasticsearch. I've been doing tons of reading, but I am hoping that the experts on SO might be able to weigh in on my cluster configuration to see if there is something that I am ...
0
votes
1answer
43 views

Best practice of adding node(s) to elasticsearch cluster

I'm using Zen Discovery module to connect all nodes together, but when I want to add a new node to the cluster, I have to change every node's discovery.zen.ping.unicast config to add the new node's ...
0
votes
1answer
12 views

Cassandra JMX doc pointers

Any good starting URL/pointers for reading up on what could be monitored through JMX on cassandra 2.0? Think off pull data into an existing ELK stack from JMX on new Cassandra clusters, but never ...
0
votes
1answer
99 views

Logstash integration with AWS Elasticsearch Service

I am using AWS Elasticsearch service to configure Elasticsearch Cluster and there is a separate server where I have installed Logstash 2.1.0 Here is my Logstash sample configuration file :- input { ...
0
votes
0answers
19 views

Minimum hardware requirements for ElasticSearch in docker [duplicate]

I am developing a search application using elasticsearch and I would like to keep the development cost as low as possible. What are the minimum requirements for elasticsearch to run properly inside a ...
-2
votes
1answer
51 views

Can Logstash, ElasticSearch and Kibana be used to log http requests?

I need something like ODBC logging on Linux. There are some older libraries for Apache and MySQL, but I do not want to use them. I found the ELK stack and installed it. Can Logstash, ElasticSearch ...
0
votes
0answers
28 views

Elasticsearch shuts down with no reason

We're running Elasticsearch 1.7.2 on ubuntu 14.04, the system has enough memory and other services on the server work just fine. But sometimes elastic shutsdown without any thing logged to it's log, ...
1
vote
0answers
69 views

Why does graylog2 delete old indicies?

According to http://docs.graylog.org/en/1.3/pages/index_model.html, graylog2 deletes old indicies based on the "elasticsearch_max_number_of_indices". Why can't graylog2 allow no. of indicies to grow? ...
1
vote
1answer
60 views

ELK stack on google compute engine

Trying to set up an ELK stack on a google compute engine server I found out that the "click to deploy" solution google is providing is outdated. None the less I tried installing Logstash and Kibana ...
0
votes
1answer
55 views

elasticsearch not creating replica shards

I am banging my head for the past two days on this one. I have setup an elasticsearch cluster of two nodes. The configuration of each node is pretty simple: node.name: "server1" ...
3
votes
1answer
69 views

Logstash - Graph IPv4 vs IPv6

I'm looking at starting to graph inbound connections to our various services (web, mail, dns etc) by IP version, ie IPv4 connections versus IPv6 connections. Currently the IPs are being stored in a ...
0
votes
0answers
56 views

Elasticsearch enable doc_values for all non-analyzed fields in template and re-index

How do I enable doc values for all non-analyzed fields in template from API ? So that indices being crated will have doc values enabled. After this how do I re-index old indices from API which will ...
0
votes
1answer
56 views

Backing Up Old Indices in Elasticsearch

We have a ELK (ElasticSearch-Logstash-Kibana) deployment in which we ship logs via logstash to Elasticsearch Cluster. Indices are created daily. We close indices which are more than 3 days old and ...
0
votes
1answer
24 views

elasticsearch: how to see that warnings are in logs / breakers are being tripped

ES 1.7.x on CentOS Our production ES cluster went down hard. We lost the entire index. Turns out, this had been in the logs for a day or more: New used memory from field ### would be larger than ...
0
votes
0answers
69 views

Is there a way to force restore an ElasticSearch snapshot with “unsupported snapshot state”

I am trying to restore a snapshot, but there were problems in some indices when we took the snapshot. We didn't get any errors when getting the backup, so we deleted the previous instance, but when we ...
1
vote
1answer
62 views

How to format log data before forwarding them as JSON to elasticsearch?

I log all events on a system to a JSON file via syslog-ng: destination d_json { file("/var/log/all_syslog_in_json.log" perm(0666) template("{\"@timestamp\": \"$ISODATE\", \"facility\": \"$FACILITY\", ...
0
votes
0answers
90 views

Fluentd : How to add custom tags to log entries sent to elasticsearch

I am quite new to Fluentd and I was wondering if there was a way to get the following logstash configuration applied to fluentd filter { grok { patterns_dir => [ ...
-1
votes
1answer
323 views

java.io.IOException: Too many open files on tomcat server

I have web application which is based on JAVA (GWT Framework). I hosted my web application on Redhat Linux Server 6.0 using Tomcat 8.0.9 in front of apache http server connected using AJP connector ...
1
vote
0answers
54 views

How to setup Kibana to work with numbers?

I try to setup a simple graph with Kibana 4.1.2 to show cumulated bytes send. From my Logstash configuration: filter { if [type] == "nginx-access" { # Field Data Type Corrections mutate { ...
-2
votes
2answers
87 views

CentOs 6.2 server with ElasticSearch crashing multiple times each day

Im running a CentOS 6.2 server 1GB Memory 98 GB Disk Size Elasticsearch version 1.4.2 with around 9000 documents and using JDBC to retrieve the records from a mysql database Over the past few ...
1
vote
0answers
29 views

Experiences with logfile monitoring

In an IT-integrator, I'm currently trying to evangelize the use of log file monitoring of hardware installations. The idea is to use a Elasticsearch/Logstash/Kibana stack (most flexibility, open ...
1
vote
0answers
166 views

How do I update Logstash data to Redshift real-time?

I am looking for a way to build a pipeline for storing the logs data (I use the ELK stack) to AWS Redshift, for analytics. I have gone through the Output plugins for ElasticSearch, and found one for ...
0
votes
1answer
141 views

How to provision dashboards into kibana?

I am giving my developers the gift of kibana using the elasticsearch that is bundled with logstash. They will have kibana in their vagrant boxes. Goal: Install and configure kibana dashboards via ...
1
vote
0answers
134 views

Curl receives empty repsonse form Elasticsearch server

I have a healthy ES cluster running on ec2. I'm tyring to connect Graylog server to the cluster using unicast, but Graylog receives an empty reponse from the unicast host. Using curl, I can ...
0
votes
1answer
25 views

Opserver Alerting that ElasticSearch Indexes and Shards are in Trouble

We're using Opserver to monitor our ElasticSearch nodes. Recently it began alerting "1 Indexes in trouble" and "3 Shards in trouble". I can't seem to find any other information on these warnings. ...
1
vote
1answer
93 views

Why does elasticsearch reports swapping when swapping is disabled?

When trying to disable JVM swapping I disabled paging on a Windows machine and rebooted: When looking at the cluster health via Elastic HQ, I see that the JVM swaps very much anyway: Why is it ...
4
votes
1answer
230 views

Why would a node not have shards?

I am setting up a small elasticsearch cluster and one of the nodes does not get any shards. It works, is part of the cluster - just empty (eu2 in the screenshot of head below): This node is a ...
0
votes
1answer
316 views

Elasticsearch index created with fieldtype mapping integer, recognized as “string” in Kibana 4

I have created an index in elasticsearch 1.7 like this: curl -XPUT 'http://localhost:9200/test' -d ' { "test" : { "properties" : { "user" : {"type" : "string", "index" : ...
1
vote
1answer
260 views

Elasticsearch Snapshot Restore with Merge

I'm experimenting with Elasticsearch in relation to backups and restoring data. I can back up data into a snapshot using curator no problems. I then physically delete the files related to the index ...
0
votes
0answers
151 views

Where does elasticsearch store graylog2 logs?

I've setup fluentd to stream docker container logs to a centralized graylog2 server. I'm hoping to setup log rotate for these logs. From what I understand, graylog stores these logs in ...
3
votes
0answers
155 views

Elasticsearch Debugging

Our elasticsearch is a mess. The cluster health is always in red and ive decided to look into it and salvage it if possible. But I have no idea where to begin with. Here is some info regarding our ...
0
votes
0answers
88 views

Elasticsearch ec2 discovery getting wrong IP?

I am running elasticsearch 1.4.3 as a worker-node/client (master=false, data=false) on AWS and I'm trying to get it to join a cluster. My elasticsearch.yml looks like this: cluster-name: ...
-1
votes
1answer
73 views

Unable to run logstash config file (permission denied) [closed]

my config file is stored in /etc/logstash/ and I ran the command $ /etc/logstash -f /etc/logstash/logstash.conf as root. However, they told me that permission denied when I tried to do that. ...
1
vote
0answers
466 views

What is wrong with my logstash configuration?

Background I have two types of log files: output from an ETL process, and then output from a downstream processor. We call these "ETL" and "Processor" logs. The ETL logs are in their own folder ...
2
votes
1answer
958 views

Logstash/elasticsearch stops accepting new data

I've set up a new proof of concept logstash system CentOS 6.6 (on Vmware 5.5) - single CPU VM with 12G RAM allocated Elasticsearch and Logstash installed from RPMs … # rpm -q elasticsearch ...
0
votes
1answer
40 views

Central Logging Options - need flat file as well as elastic search

I would like to have ephemeral ec2 instances push logs to a central flat-file store for archiving and manual perusing, as well has have that data pushed to elastic search. Is there a single agent ...
0
votes
0answers
227 views

Elastic Search - Failed to execute while moving to second phase

I am trying to get an ELK stack working in Ubuntu 14.04 and get this error. This is not a memory issue like most of the posts i've read, the machine has loads free. I get this error when I open the ...