Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

Filter by
Sorted by
Tagged with
37 votes
6 answers
54k views

How to increase disk size in a stateful set

I'm managing an Elasticsearch deployment in Kubernetes. I see that the disk storage is getting close to full, so I would like to increase the size of the persistent volumes. I want to change this ...
Brent Bradburn's user avatar
24 votes
1 answer
79k views

How to check ElasticSearch version in Linux [closed]

I am just curious on how to check ElasticSearch version in Linux?
Aminah Nuraini's user avatar
20 votes
4 answers
80k views

Ubuntu uninstall elasticsearch

I installed elasticsearch.90.7 with a deb file in ubuntu. I tried to uninstall elasticsearch.90.7 with this command: sudo apt-get --purge autoremove elasticsearch And then I downloaded elasticsearch-...
alireza's user avatar
  • 313
18 votes
2 answers
21k views

Is there a way to stop the AWS Elasticsearch?

I setup an AWS Elasticsearch Domain recently but I didn't see a way to stop it (like you can with an EC2 instance), which means I'm continuously billed. At this stage I just need to do some testing ...
codecraig's user avatar
  • 387
16 votes
1 answer
14k views

Scaling Logstash (with redis/elasticsearch)

Over a cluster of over 12 centos 5.8 servers, I deployed logstash using the native logstash shipper, which sends /var/log/*/*.log back to a central logstash server. We tried using rsyslogd as the ...
Tom O'Connor's user avatar
  • 27.6k
16 votes
3 answers
40k views

Unable to change vm.max_map_count for elasticsearch

Prehistory I have elasticsearch and SugarCRM7 running on CentOS 6.5. Every day I face the same problem: java outOfMemory error. That happens because of small vm.max_map_count value, 65530 only when ...
Valentina's user avatar
  • 161
13 votes
1 answer
37k views

Elasticsearch is using way too much disk space

I have a CentOS 6.5 server on which I installed Elasticsearch 1.3.2. My elasticsearch.yml configuration file is a minimal modification of the one shipping with elasticsearch as a default. Once ...
mac's user avatar
  • 500
10 votes
1 answer
7k views

Recommendations for Elastic Search hardware [closed]

Are there any good guides for hardware level to support ElasticSearch? Are recommendations for Lucene or Solr a good place to start? We're looking at rolling out a deployment starting with 27 million ...
James Socol's user avatar
9 votes
1 answer
485 views

Elasticsearch dies when Logstash attempts to write data

I've got a Raspberry Pi 2 (latest Raspbian as of Apr 2015) setup that last week was running both ElasticSearch and Logstash on a test network (not a straightforward setup, but it was stable for over a ...
anyweez's user avatar
  • 193
8 votes
4 answers
5k views

How best to monitor logstash?

I've seen this question on the mailing list a few times but haven't had a satisfactory answer. How best to monitor that the pipeline isn't stuck? Clients -> logstash -> elasticsearch. Logstash and ...
Dan Garthwaite's user avatar
7 votes
3 answers
16k views

Elastic Search listening only on IPv6 [CentOS]

I've installed Elastic Search 1.7.4 on CentOS 7.2 this way: wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.4.noarch.rpm sudo rpm -ivh elasticsearch-1.7.4.noarch.rpm ...
miloske85's user avatar
7 votes
2 answers
22k views

How to install Elasticsearch plugins with docker container?

I am a newbie in Docker, I want to install plugins in my container of elasticsearch, in this case they are: analysis-icu analysis-phonetic I know that in a traditional way it would be like this: /...
Alex's user avatar
  • 173
7 votes
3 answers
7k views

Secure logstash and elasticsearch

I'm considering running logstash on my prod server (simple install. http://logstash.net/docs/1.1.13/tutorials/getting-started-simple) and set kibana to access logs. My concern is: how to secure my ...
CoBaLt2760's user avatar
7 votes
4 answers
26k views

Error Could not contact Elasticsearch at http://localhost:9200. - kibana dashboard

'Error Could not contact Elasticsearch at http://localhost:9200. Please ensure that Elasticsearch is reachable from your system.' Currently I am following http://packetbeat.com/getstarted I ...
druuu's user avatar
  • 171
6 votes
1 answer
10k views

How well does elasticsearch compress data?

I am looking to scope the servers required for an elasticsearch proof-of-concept. Ultimately, my question is this: Given 1GB of json text indexed by elasticsearch, how much can I expect to be ...
cmonkey's user avatar
  • 163
6 votes
1 answer
670 views

Different linux page cache behavior for servers doing the same work

I have two sets of servers w/128G memory, distinguished by when they were provisioned, that are behaving very differently while running the exact same daemon (elasticsearch). I am using elasticsearch ...
Erik B's user avatar
  • 91
6 votes
2 answers
1k views

Sending Subversion logs to Logstash

My requirement is to send subversion logs(i.e username,revision number...) to logstash for parsing(then store it in elastic search and finally displayed it via kibana).Since subversion use its own ...
Prashant Lakhera's user avatar
6 votes
1 answer
11k views

ElasticSearch Server Randomly Stops Working

I have 2 ES servers that are being fed by 1 logstash server and viewing the logs in Kibana. This is a POC to work out any issues before going into production. The system has ran for ~1 month and every ...
Eric's user avatar
  • 1,393
6 votes
0 answers
409 views

Is there disk based file system with dedicated memory caching for Linux? [closed]

We have two (or more sometimes) applications battling for the OS page cache. In particular ElasticSearch is very dependent on the OS page cache. To the point that cache misses can make or break ...
Andrew White's user avatar
5 votes
2 answers
7k views

What is the purpose of elasticsearch logs? How to manage them?

As I understand indexes(or data) are being stored in /var/lib/elasticsearch by default, this folder contains nodes with 0 and 1 folders and overall size of these folders is 376M. The logs are ...
Rustam Gasanov's user avatar
5 votes
1 answer
2k views

Why would a node not have shards?

I am setting up a small elasticsearch cluster and one of the nodes does not get any shards. It works, is part of the cluster - just empty (eu2 in the screenshot of head below): This node is a worker, ...
WoJ's user avatar
  • 3,657
5 votes
3 answers
12k views

Ubuntu 15.04: Elasticsearch service doesn't start

I followed the first part from the tutorial here to install Elasticsearch. I didn't install java because I already have that installed with other java programs running. sudo service elasticsearch ...
MadeOfAir's user avatar
  • 201
5 votes
2 answers
6k views

How to index MySQL from ElasticSearch

Im trying to add some full text searching to my LAMP web application. Ive come across ElasticSearch and I like what it has to offer. I have read the guides and understand how to index a single item by ...
CraigH's user avatar
  • 195
5 votes
1 answer
27k views

What does "GC (Allocation Failure)" mean in my ElasticSearch 5.6 logs?

I see these constantly in my Elasticsearch 5.6.3 logs. Is this a signal that I should scale up and add more RAM? Or is this just normal operations for ElasticSearch? [GC (Allocation Failure) [...
Antonius Bloch's user avatar
5 votes
2 answers
2k views

How to secure an Internet-facing Elastic Search implementation in a shared hosting environment?

(Originally asked on StackOverflow, and recommended that I move it here) I've been going over the documentation for Elastic Search and I'm a big fan and I'd like to use it to handle the search for my ...
casperOne's user avatar
  • 151
5 votes
2 answers
5k views

"Index Patterns: Please specify a default index pattern" in Kibana

I'm trying to create a simple hello world for ELK and be able to see kibana reports via the internet. I've installed kibana, logstash, nginx and elastic search. Here's my /etc/logstash/conf.d/10-...
Dett's user avatar
  • 53
5 votes
0 answers
3k views

elasticsearch using bulk insert to post JSON file

I have the following JSON file I have used awk to get rid of empty spaces, trailing, next line awk -v ORS= -v OFS= '{$1=$1}1' data.json I have added a create request at the top of my data.json ...
user1007727's user avatar
5 votes
1 answer
780 views

Elasticsearch Debugging

Our elasticsearch is a mess. The cluster health is always in red and ive decided to look into it and salvage it if possible. But I have no idea where to begin with. Here is some info regarding our ...
Beginner's user avatar
  • 211
4 votes
4 answers
32k views

Nothing appearing in kibana dashboard

I have installed Logstash ElasticSearch Kibana on an EC2 instance. I can visit http://example.com:9200 Here I get { "status" : 200, "name" : "Aleta Ogord", "version" : { "number" : "1.1.0"...
user2099762's user avatar
4 votes
3 answers
9k views

Logstash / Elasticsearch - trasform fields to lowercase in output conf

I have a standard ELK stack currently storing numerous log outputs. I'm trying to separate my indices to be source-specific. As part of my FileBeats config, some standard fields are always generated ...
m8r-3wo9bu's user avatar
4 votes
1 answer
889 views

ElasticSearch snapshot backup sliding expiration - possible?

I'm planning on using the ElasticSearch S3 cloud plugin to create snapshots of our ES cluster. This all looks fairly straight forward but I'm wondering whether its possible integrate it into our ...
justcompile's user avatar
4 votes
4 answers
546 views

Securing elasticsearch

There appears to be a serious lack of information on this despite the fact that as installed elasticsearch is extremely exploitable. My main fear in using it is that as a non-expert I have no idea ...
George Reith's user avatar
4 votes
1 answer
3k views

Adding TTL to elasticsearch with logstash

I can't figure out how to apply this documentation http://www.elasticsearch.org/guide/reference/mapping/ttl-field/ to logstash. Specifically, i don't know which keys to use (instead of "tweet" etc). ...
Maciej Swic's user avatar
4 votes
1 answer
2k views

Logstash/elasticsearch stops accepting new data

I've set up a new proof of concept logstash system CentOS 6.6 (on Vmware 5.5) - single CPU VM with 12G RAM allocated Elasticsearch and Logstash installed from RPMs … # rpm -q elasticsearch logstash ...
Paul Haldane's user avatar
  • 4,547
4 votes
1 answer
2k views

Elasticsearch: High traffic on loopback interface

One of our application uses Elasticsearch (1.4.4) as an in-memory cache. The application is a Java webapp deployed on Tomcat 7 with Oracle 1.7. The elasticsearch instance is a one-node setup deployed ...
ercpe's user avatar
  • 576
4 votes
1 answer
248 views

elasticsearch server is unreachable every 2 hours

This question is related to this one. We now know that the errors come from elasticsearch. The problems are still not resolved despite the modifications and optimizations made on the es instance. ...
COil's user avatar
  • 229
4 votes
2 answers
3k views

Elasticsearch performance tuning

In a Single Node Elastic Search along with logstash, We tested with 20mb and 200mb file parsing to Elastic Search on Different types of the AWS instance i.e Medium, Large and Xlarge. Logstash conf ...
Devaraj's user avatar
  • 41
4 votes
3 answers
10k views

Foward slash in kibana 3 query

I'm trying to add a query that will match a request that ends with a slash, like this one: n.n.n.n - - [16/Oct/2013:16:40:41 +0100] "GET / HTTP/1.1" 200 25058 "-" "Mozilla/5.0 (iPad; CPU OS 7_0_2 ...
G Mawr's user avatar
  • 173
3 votes
3 answers
470 views

Simple full-text search server

I have been looking at search solutions like Sphinx, Solr and Elasticsearch but they are all way too complex for what I need. I'm basically looking for a server software, best distributed, that ...
thwd's user avatar
  • 177
3 votes
3 answers
3k views

Must all nodes of an Elasticsearch cluster be identical?

In an Elasticsearch cluster, must all nodes be identical i.e. having the same: Operating System Amount of RAM CPU speed and number of cores etc. or is it allowed for the nodes to be different? (...
dr_'s user avatar
  • 1,105
3 votes
2 answers
11k views

No route to host elasticsearch exception

When turning on elasticsearch, I am getting the following configuration error: [2015-04-21 20:49:45,635][INFO ][discovery.zen ] [Blackwulf] failed to send join request to master [[...
580farm's user avatar
  • 251
3 votes
1 answer
2k views

Migrating to AWS Cloud with auto-scaling - where to put Redis and ElasticSearch?

I've been trying to research this topic but haven't found anywhere that recommends where to install services such as Redis and ElasticSearch when migrating to a cloud framework. I'm currently running ...
RobMasters's user avatar
3 votes
1 answer
7k views

vm.max_map_count problems on GKE ElasticSearch StatefulSet

A problem appeared on working ElasticSearch cluster on GKE. Nodes with "data" roles began to crash unexpectedly with an error: max virtual memory areas vm.max_map_count [65530] is too low, increase ...
Victor Bessonov's user avatar
3 votes
1 answer
3k views

Logstash integration with AWS Elasticsearch Service

I am using AWS Elasticsearch service to configure Elasticsearch Cluster and there is a separate server where I have installed Logstash 2.1.0 Here is my Logstash sample configuration file :- input { ...
Siddharth Sharma's user avatar
3 votes
2 answers
1k views

Full data backup to Amazon S3

I have an Ubuntu server hosted on Digital Ocean that has outgrown it's existing backup solution. The relevant parts of the stack I use are Node.js, MongoDB, and Elasticsearch. So far backups have ...
A.M.K's user avatar
  • 163
3 votes
1 answer
3k views

How to rename and join an elasticsearch server preserving data?

I tested elasticsearch and decided to make a cluster. To this I renamed the node and defined a cluster cluster.name = mycluster node.name = firstnode When restarting the changes are taken into ...
WoJ's user avatar
  • 3,657
3 votes
1 answer
3k views

Why is this exclude_lines in filebeat excluding all logs?

I'm using ELK Stack, and I've got it working pretty well for most of my servers. The exception is that I have a gitlab server that has a ping to/from a gitlab-ci server that happens in the gitlab-...
trueCamelType's user avatar
3 votes
1 answer
12k views

elk stack error "unable to fetch mapping do you have indices matching the pattern"

i am trying to setup ELK stack with collectd on Ubuntu 16.04 LTS (so pretty much latest version of stack available) kibana is behind nginx proxy (followed this guide https://www.digitalocean.com/...
uberrebu's user avatar
  • 513
3 votes
1 answer
3k views

Using Logstash as shipper?

We are shipping logs from servers and using THE logstash on each server for shipping. So we read logs from the glob "/root/Desktop/Logstash-Input/**/*_log". input { file{ ...
Siddharth Trikha's user avatar
3 votes
1 answer
5k views

Logstash Server Log FIle Flooded With "Failed to flush outgoing items" Error Messages

I'm new to Logstash, trying to get it running following the tutorial in "The Logstash Book". On page 44, the guide suggests to tail the Logstash process' main log file called central.log. After ...
Josh's user avatar
  • 31

1
2 3 4 5
8