Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

learn more… | top users | synonyms

1
vote
1answer
21 views

Logstash/elasticsearch stops accepting new data

I've set up a new proof of concept logstash system CentOS 6.6 (on Vmware 5.5) - single CPU VM with 12G RAM allocated Elasticsearch and Logstash installed from RPMs … # rpm -q elasticsearch ...
0
votes
1answer
18 views

Central Logging Options - need flat file as well as elastic search

I would like to have ephemeral ec2 instances push logs to a central flat-file store for archiving and manual perusing, as well has have that data pushed to elastic search. Is there a single agent ...
0
votes
0answers
27 views

Elastic Search - Failed to execute while moving to second phase

I am trying to get an ELK stack working in Ubuntu 14.04 and get this error. This is not a memory issue like most of the posts i've read, the machine has loads free. I get this error when I open the ...
0
votes
2answers
94 views

ELK stack: Can't load Kibana 4

I have installed Elasticsearch, Logstash, Kibana and Nginx following this guide, exept I used the latest version of everything. ...
0
votes
0answers
20 views

Elasticsearch questions

After a month of testing the ELK stack I'm planning to move it on production. Now I'm facing a few dillemas. The plan is to index around 30GB of data per day (mainly log files) and my questions are: ...
0
votes
0answers
20 views

Fetch logstash-forwarded auth logs from elasticsearch database

In my logstash-forwarder, i have configured to ship /var/log/auth.log messages under the files section as, { "paths": [ "/var/log/syslog", "/var/log/auth.log" ], ...
1
vote
1answer
170 views

Kibana 4, Logstash dashboard: how do I require Nginx authentication when saving but allow anonymous views?

I would like to require auth_basic nginx authentication to save all kibana 4 dashboards but allow anyone to view dashboards without authentication. I recently installed an ELK (Elasticsearch 1.4.5, ...
2
votes
2answers
179 views

Full data backup to Amazon S3

I have an Ubuntu server hosted on Digital Ocean that has outgrown it's existing backup solution. The relevant parts of the stack I use are Node.js, MongoDB, and Elasticsearch. So far backups have ...
0
votes
1answer
33 views

Logstash is not processing MySQL logs correctly

As you can see, the message is not just the message, but also contains the date and timestamp. . . This is the MySQL Log: /var/log/mysql/error.log 150630 9:01:29 [Warning] Access denied for user ...
0
votes
0answers
77 views

Sending log4j logs to ELK using SocketAppender

ElasticSearch crashes when it tries to parse the logs received from Tomcat using SocketAppender. Example error is below: org.elasticsearch.index.mapper.MapperParsingException: failed to parse ...
0
votes
1answer
73 views

Elasticsearch network.bind_host and network.publish_host

I have a 4gb ram machine that works as a DB server for a ios/webapp. On that machine I only have Elasticsearch and MySql installed. What I want to do now is to only allow connections to ES from two ...
0
votes
2answers
266 views

Ubuntu uninstall elasticsearch

I installed elasticsearch.90.7 with a deb file in ubuntu. I tried to uninstall elasticsearch.90.7 with this command: sudo apt-get --purge autoremove elasticsearch And then I downloaded ...
0
votes
0answers
248 views

Unable to fetch mapping. Do you have indices matching the pattern?

I am new to ELK and recently installed ELK on Windows Server 2008 and on Windows 7 machine. But on both the places if I open Kibana I get the message: "Unable to fetch mapping. Do you have indices ...
1
vote
1answer
75 views

Setting memory mapped file limit for Windows?

Hi running Windows Server 2008 R2 Enterprise. My server has 128GB of RAM. I'm using Elasticsearch which is configured with ES_HEAP_SIZE=30GB With Task Manager Private Working Set: 33GB Working Set: ...
1
vote
1answer
248 views

Ubuntu 15.04: Elasticsearch service doesn't start

I followed the first part from the tutorial here to install Elasticsearch. I didn't install java because I already have that installed with other java programs running. sudo service elasticsearch ...
0
votes
0answers
39 views

Separating Elasticsearch read and write

Is it possible to build a 2 or 3 node Logstash/ES cluster that separates read/query and write/indexing, so complex queries don't impact log indexing? Reading here: ...
1
vote
0answers
65 views

Elasticsearch snapshot / curl error

I'm currently having an issue getting backups for ElasticSearch working correctly via their snapshotting system. Per their instructions, a snapshot repository has to be configured first, using this ...
0
votes
1answer
44 views

Stopping an Elasticsearch snapshot

We do nightly snapshots to S3. They normally take ~2hrs. I have one that has been running for 4 days now. I've tried to stop it with this command: time curl -XDELETE -s ...
0
votes
1answer
318 views

Elastic Search Unassigned Shards

my cluster state was green and after restart of service one shard remains UNASSIGNED and status goes to yellow. I have 2 machines with 5 shard and 1 replica set settings. I am using default config ...
4
votes
1answer
175 views

Elasticsearch: High traffic on loopback interface

One of our application uses Elasticsearch (1.4.4) as an in-memory cache. The application is a Java webapp deployed on Tomcat 7 with Oracle 1.7. The elasticsearch instance is a one-node setup deployed ...
0
votes
1answer
107 views

Is it possible to access Kibana (Elasticsearch) via ssh tunnel?

I am trying to acces kibana from a remote machine through an SSH tunnel: $ ssh -f user@elastichost -L 5601:elastichost:5601 -N My remote browser just timesout waiting for a response. Is there a ...
2
votes
0answers
50 views

Shutting down elasticsearch completely?

I have found a number of posts about shutting down a node or a cluster of nodes, but what about stopping all elasticsearch processes on a machine? Even after shutting down all nodes (curl -XPOST ...
1
vote
0answers
53 views

IDS (or similar) to query ELK logs and send alarms on strange situations [closed]

I have an ELK stack (that receives logs from mostly web services,web applications). I'd like to not only store and visualize some stats about the logs(features of ELK) but also to use them to detect ...
1
vote
2answers
630 views

No route to host elasticsearch exception

When turning on elasticsearch, I am getting the following configuration error: [2015-04-21 20:49:45,635][INFO ][discovery.zen ] [Blackwulf] failed to send join request to master ...
1
vote
1answer
240 views

Elasticsearch service often goes down or gets killed

My webserver is running ubuntu 14.10 with elasticsearch 1.5.0 and java 1.7u55 For some reason, the elasticsearch service often goes down, resulting in my website not being available to my users ...
1
vote
0answers
98 views

elasticsearch glibc memory corruption

My elastic search clusters keep crashing for no reason. All I can see in the logs is : INFO | jvm 4 | 2015/04/17 19:10:09 | *** glibc detected *** /usr/java/jre1.7.0_04/bin/java: malloc(): ...
2
votes
0answers
171 views

elasticsearch fails periodically on ubuntu with systemd : how to resolve this?

I am using Elasticsearch on Ubuntu 14.10 and it's periodically going down (entering a failed state). When down, this throws exceptions on my website, users can't search, connect or update database ...
0
votes
1answer
51 views

Secure elasticsearch but allow remote CRUD

I'm going to run a bit of software on server A to add and remove entries from elasticsearch on server B. How can I allow these remote create and delete operations, but still keep ES secure? Is ...
0
votes
0answers
59 views

ElasticSearch versus arcsight benchmark comparisons

For those of us looking into huge SIEM implementations, are there any datasheets comparing the benchmarks of arcsight with elastic search. Such benchmarks would include the following: given the same ...
2
votes
0answers
567 views

Unable to change vm.max_map_count for elasticsearch

Prehistory I have elasticsearch and SugarCRM7 running on CentOS 6.5. Every day I face the same problem: java outOfMemory error. That happens because of small vm.max_map_count value, 65530 only when ...
2
votes
0answers
97 views

Elasticsearch dies when Logstash attempts to write data

I've got a Raspberry Pi 2 (latest Raspbian as of Apr 2015) setup that last week was running both ElasticSearch and Logstash on a test network (not a straightforward setup, but it was stable for over a ...
0
votes
0answers
65 views

Elastic search custom mysql server

I want to use ElasticSearch nodes with Amazon RDS servers. How I can do this? The idea is to have the ElasticSearch feature only with a remote DB.
2
votes
0answers
137 views

Benchmarking for DISK I/O Writes Per Second - ElasticSearch and MySQL

During working on our project we needed to decide should we use "ElasticSearch" or "MySQL" to record the audit/trail logs of the system. Searching speed is not an issue here, we needed to see the disk ...
0
votes
0answers
36 views

elasticsearch not enabled when installed on additional device

Here's the deal : I'm working with AWS EC2 instances (Centos 7). Elasticsearch is installed from the sources in the folder /app. The elasticsearch service is managed by init.d. The problem is that if ...
0
votes
2answers
189 views

elasticsearch-river-jdbc inserting duplicate record in mysql db

Sorry I am newbie to Elasticsearch and I am using elasticsearch-river-jdbc to connect to myql database,everything is working fine except for the fact that everytime it run as per schedule its ...
0
votes
1answer
106 views

Is it generally a bad idea to run MongoDB and ElasticSerach on the same nodes

Simple question: is it stupid to run mongodb and elasticsearch on the same nodes within a cluster of three if you take care of mongodb's memory hunger using cgroups and ignore the aspect of concurring ...
2
votes
0answers
93 views

Using database for storing text files and images [closed]

I need to build a image/static file cache which would host maybe few hundred up to thousand static files from 1kb to ~200kb. I'd need super fast reads (I'm using redis, but it's one threaded and I'm ...
0
votes
0answers
314 views

Install logstash forwarder

Any body please help me how to install logstash forwarder in 12.04 ubuntu version? I followed logstash book to install forwarder I got some ruby error these are the steps i followed $ sudo ...
0
votes
0answers
58 views

What the best practices to import Elasticsearch indices

I am running an elasticsearch cluster of 5 nodes in a production subnetwork, it's a "front" cluster with lot of search/read operations (DNS es-1.data.prod, es-2.data.prod and so on). In an other side, ...
5
votes
2answers
248 views

Sending Subversion logs to Logstash

My requirement is to send subversion logs(i.e username,revision number...) to logstash for parsing(then store it in elastic search and finally displayed it via kibana).Since subversion use its own ...
2
votes
1answer
567 views

Elasticsearch performance tuning

In a Single Node Elastic Search along with logstash, We tested with 20mb and 200mb file parsing to Elastic Search on Different types of the AWS instance i.e Medium, Large and Xlarge. Logstash conf ...
0
votes
0answers
80 views

How parse a multiple objects serialized in log with logstash and grok

I'm parsing some log files with grok, but I have the following trace: [es.org.some.Object[ attr1=53212 searchTags=[love, rock] ]] [es.org.some.Object[ attr1=54612 searchTags=[love, peace, rock] ...
6
votes
0answers
93 views

Is there disk based file system with dedicated memory caching for Linux? [closed]

We have two (or more sometimes) applications battling for the OS page cache. In particular ElasticSearch is very dependent on the OS page cache. To the point that cache misses can make or break ...
0
votes
1answer
770 views

Error while setting up grafana with graphite and nginx

While trying to setup grafana with graphite using nginx server. I am getting this error: No 'Access-Control-Allow-Origin' header is present on the requested resource. I tried changing configuration ...
2
votes
4answers
3k views

Error Could not contact Elasticsearch at http://localhost:9200. - kibana dashboard

'Error Could not contact Elasticsearch at http://localhost:9200. Please ensure that Elasticsearch is reachable from your system.' Currently I am following http://packetbeat.com/getstarted I ...
1
vote
2answers
132 views

Elasticsearch index much larger than the actual size of the logs it indexed?

I noticed that elasticsearch consumed over 30GB of disk space over night. By comparison the total size of all the logs I wanted to index is only 5 GB...Well, not even that really, probably more like ...
0
votes
0answers
64 views

ElasticSearch snapshot backup sliding expiration - possible?

I'm planning on using the ElasticSearch S3 cloud plugin to create snapshots of our ES cluster. This all looks fairly straight forward but I'm wondering whether its possible integrate it into our ...
-1
votes
1answer
312 views

Why doesn't logstash grab or index the files from the mapped drive?

I don't understand why logstash is so finicky with network resources. I shared a folder on another machine and then mapped it as Z: under Windows Explorer. I've verified the path and everything. I can ...
2
votes
1answer
624 views

How to rename and join an elasticsearch server preserving data?

I tested elasticsearch and decided to make a cluster. To this I renamed the node and defined a cluster cluster.name = mycluster node.name = firstnode When restarting the changes are taken into ...
0
votes
2answers
733 views

What is the purpose of elasticsearch logs? How to manage them?

As I understand indexes(or data) are being stored in /var/lib/elasticsearch by default, this folder contains nodes with 0 and 1 folders and overall size of these folders is 376M. The logs are ...