Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

3
votes
1answer
858 views

Why is this exclude_lines in filebeat excluding all logs?

I'm using ELK Stack, and I've got it working pretty well for most of my servers. The exception is that I have a gitlab server that has a ping to/from a gitlab-ci server that happens in the gitlab-...
0
votes
0answers
285 views

centos dedicated server backup strategy for mongodb and elasticsearch db

I have a main server (dedicated CentOS 7, XFS file system, 2X1TB SSD, software RAID1) connected via LAN to my backup server (dedicated CentOS 7, XFS file system, 2X2TB HDD, software RAID1) in my setup....
0
votes
1answer
721 views

Using HAProxy for syslog traffic at scale

I have an environment that generates a frankly ludicrous amount of Syslog traffic - this is mostly due to a culture of leaving debug-level logging on in production applications (and changing this ...
0
votes
1answer
2k views

packet_write_wait: Connection to 10.0.0.8 port 22: Broken pipe after sudo service elasticsearch start

I'm getting the weirdest error. After running config management to install elasticsearch on an instance, when I do sudo service elasticsearch start I get a Broken pipe error and ssh cuts out. Then ...
0
votes
1answer
950 views

Elasticsearch: no JDK found… but JAVA_HOME seems set properly

When I'm trying to install elasticsearch (Ubuntu 16.04) I get the error message: elasticsearch[15885]: * no JDK found - please set JAVA_HOME But I have only one Java and I think JAVA_HOME is set ...
13
votes
3answers
24k views

Unable to change vm.max_map_count for elasticsearch

Prehistory I have elasticsearch and SugarCRM7 running on CentOS 6.5. Every day I face the same problem: java outOfMemory error. That happens because of small vm.max_map_count value, 65530 only when ...
3
votes
1answer
2k views

New ElasticSearch 5.2.1 Install on Ubuntu 16.04: Connection refused

I just followed this install guide for Elasticsearch via APT: https://www.elastic.co/guide/en/elasticsearch/reference/5.0/deb.html When I run curl -X GET 'http://127.0.0.1:9200/' (or localhost) I ...
18
votes
1answer
53k views

How to check ElasticSearch version in Linux [closed]

I am just curious on how to check ElasticSearch version in Linux?
0
votes
1answer
43 views

elasticsearch network recommendations server linux

I am looking for assistance regarding the network settings of my two servers that I am setting up for elasticsearch. On each server is an embedded network card and a extra network card is install with ...
0
votes
1answer
36 views

Different elasticsearch nodes for different use cases

Our Elasticsearch cluster is used to provide search results for a frontend. Most of the traffic is pretty negligible and the cluster can handle the load just fine. At a scheduled time each week, ...
5
votes
1answer
609 views

Elasticsearch Debugging

Our elasticsearch is a mess. The cluster health is always in red and ive decided to look into it and salvage it if possible. But I have no idea where to begin with. Here is some info regarding our ...
0
votes
1answer
616 views

Info sent from Logstash via elastic output not showing in Kibana, but file output works fine - what am I doing wrong?

I have an "ELK stack" configuration and, at first, was doing the standard 'filebeat' syslog feeding from logstash with the elasticsearch output plugin. It worked just fine. Now I have added a TCP ...
0
votes
2answers
379 views

ElasticSearch doesn't start after installing 5.6.4

I just installed ElasticSearch 5.6.4 from https://www.elastic.co/downloads/elasticsearch via DEB (I use Debian 9) package using these commands: wget https://artifacts.elastic.co/downloads/...
1
vote
1answer
2k views

Could not access pid file for elasticsearch

I am not a die-hard linux engineer, but I am having some problems with starting Elastic Search. Some stats: Server: Ubuntu Server 11.04 Elastic Search: 1.2.3 (installed with appitude) When I start ...
0
votes
0answers
550 views

Automatically set Java Heap Size for ElasticSearch on Linux

I am trying to automatically set the heap size on my ElasticSearch v5.6.3 nodes (Ubuntu 16) The machines are hosted on Azure and I want to do this so that when I scale up the machine, it ...
2
votes
1answer
2k views

Stopping an Elasticsearch snapshot

We do nightly snapshots to S3. They normally take ~2hrs. I have one that has been running for 4 days now. I've tried to stop it with this command: time curl -XDELETE -s localhost:9200/_snapshot/sim-...
5
votes
1answer
294 views

Different linux page cache behavior for servers doing the same work

I have two sets of servers w/128G memory, distinguished by when they were provisioned, that are behaving very differently while running the exact same daemon (elasticsearch). I am using elasticsearch ...
0
votes
1answer
196 views

Accessing remote kibana/ES from client : time out

I am trying to access kibana/elastic search on a VPS from my computer. On the VPS if I curl -XGET 'http://SERVERIP:9200' I get a good answer from ES, and same with Kibana. On my computer, if I use ...
0
votes
0answers
436 views

Who should own /usr/share/elasticsearch/ in order to Elasticsearch to start

I'm starting with elasticsearch and noticed that in order to startup elasticsearch the directory /usr/share/elasticsearch/ and all subs need to be own by elasticsearch:elasticsearch. If it's own by ...
0
votes
1answer
394 views

Elastic Search status suddenly yellow

Suddenly ES won't start properly anymore. I have no clue why. Tried 2.3.4 Tried 5.6.1 (fresh download!) Tried 5.6.1 via Docker Deleted my indexes When I start ES (2.3.4) I get this: Cluster health ...
1
vote
1answer
821 views

How to check if network traffic on a specific port is secured/encrypted

We are running Elasticsearch 5.5.1 on Windows 2008 R2. We have installed Searchguard SSL plugin to enable SSL on Elasticsearch's transport port(9300) and are using jks certificate provided by our ...
0
votes
0answers
88 views

Bitbucket fails to start on Ubuntu 16.04 LTS

I installed Bitbucket 5.3.1 from the terminal, but when I attempt to startup the app it hangs with the following log output: INFO [spring-startup] c.a.s.i.hazelcast.HazelcastLifecycle Updating ...
0
votes
1answer
72 views

Why var/log/elasticsearch is deleted if I reboot ubuntu 16.04?

I am new to elasticsearch and figuring out how to setup ELK stack for logging syslog. I successfully installed all ELK stacks and it works fine. However, after I installed ELK stack, when I reboot, ...
3
votes
1answer
4k views

Is there a way to stop the AWS Elasticsearch?

I setup an AWS Elasticsearch Domain recently but I didn't see a way to stop it (like you can with an EC2 instance), which means I'm continuously billed. At this stage I just need to do some testing ...
1
vote
3answers
4k views

Date field is being presented as integer

I have an Elastic Stack server (on Hyper-v) that is ingesting data via a logstash exec command and performing analytics on it. Everything is working great except a date field that is being displayed ...
0
votes
1answer
250 views

Logstash creating many tcp connections for single hosts

Logstash: 5.3.0 nxlog: nxlog-ce-2.9.1716 Checking nestat I have like 300 established connections even though I have only about 50 hosts. I have like 40 connections for some hosts when I the client's ...
1
vote
1answer
193 views

Logstash / Elasticsearch - Balancing index count with performance

We have a 4 data node ElasticSearch cluster: each node has 4 cores, 16GB RAM, and 160GB storage (the cluster has separate dedicated master nodes). The cluster is responsible for storing and ...
1
vote
0answers
70 views

Elasticsearch: fine-tuning for RAID0 HDD array storage

It is usually recommended to have the following index settings in case spinning disks are used for Elasticsearch data storage: index.store.throttle.type: none index.merge.scheduler.max_thread_count: ...
0
votes
1answer
362 views

Ubuntu 16.04 & Elasticsearch 1.7 - Startup Error

I have recently added a new node(fresh install of ubuntu & elastic install via dpkg) to my existing cluster, and randomly one server would not start elasticsearch on a restart. When manually ...
0
votes
2answers
938 views

Troubleshooting rsyslog integration with ELK stack

I'm trying to configure rsyslog to send logs to logstash on CentOS. So I was following the tutorial. However, after setting up, nothing happens. Everything started ok, not error occurred but no logs ...
0
votes
1answer
501 views

Logstash output to Azure blobstorage

I am currently importing IIS-logs into Logstash using Filebeat, and the Logstash is configured to output the documents into Elasticsearch. Now I also want to output my IIS logs to Azure storage (blob) ...
0
votes
1answer
384 views

Applying item template to elasticsearch

I am trying to apply an item template to my elasticsearch cluster, to deal with the issue of having fields with content longer than 32kb. I am using version 2.4.4, as this is the highest supported ...
1
vote
1answer
456 views

Using ELK X-pack for general purpose alerts and alarms

The X-pack package from Elastic is fully integrated with Elasticsearch and Kibana to provide (among other things) an alarm reporting platform. I believe that the regular use case is to build that ...
2
votes
1answer
678 views

How to set max file descriptors for a container?

I want to run elasticsearch in a container (namely nspawn) and it requires the number of max file descriptors to be higher than the default 1024. This is done, in the container, in the systemd ...
1
vote
2answers
6k views

Problems configuring logstash for email output

I'm trying to configure logstash to send email alerts and log output in elasticsearch / kibana. I have the logs successfully syncing via rsyslog, but I get the following error when I run /opt/...
0
votes
1answer
633 views

Benefits of running an Elasticsearch cluster on a single (docker) host

I'm currently running an Elasticsearch v2.3 cluster (using docker containers) on a single host. (1 gateway, 3 master nodes and 3 data nodes) In the near future I will be upgrading towards ...
0
votes
1answer
598 views

docker-compose ELK container fails to start

I am deploying a docker-compose stack of 5 applications on a single AWS EC2 host with 32GB RAM. This includes: 2 x Java containers, 2 x Rails containers, 1 x Elasticsearch/Logstash/Kibana (ELK) ...
0
votes
1answer
408 views

How filter already collected info in elasticsearch5.2 on Ubuntu 16.04

Good Evening. I have a ELK stack as follows: Clients with logbeat (windows 2003, 2008 2012, and Linux Ubuntu 16.04) logstash (FreeBSD 11.0) elasticsearch5.2/kibana/nginx 10(Ubuntu 16.04) The problem ...
0
votes
2answers
553 views

Is it safe to delete Gralog Indexes

ElastiCache Cluster was in red state. Below is the out of cluster health Status { "cluster_name" : "graylog", "status" : "red", "timed_out" : false, "number_of_nodes" : 1, "number_of_data_nodes" : 1, ...
0
votes
2answers
2k views

Ubuntu 16 server, systemctl restart elasticsearch does nothing, and there's no log

On ubuntu 16 lts server system, a newly installed elasticsearch instance won't start up with systemctl restart elasticsearch (or with start). No logs are created, manually running /usr/share/...
2
votes
1answer
2k views

elastichsearch node health check for haproxy

I have place haproxy in front of a three node ES(elasticsearch) cluster. So far the way i check for each node in haproxy is by using httpcheck. Bellow is a snippet of my config: backend elastic_nodes ...
0
votes
2answers
862 views

How can I check if my ElasticSearch process is using compressed oops?

I'd like to figure out if I need to tune my currently-running ElasticSearch cluster. How do I check if a running HotSpot process is using compressed oops or not?
0
votes
1answer
2k views

Elasticsearch Delete Index with Special Characters

Unfortunately i created an Index in Elasticsearch with the name: "%{[@metadata][beat]}-2016.11.17" Any Idea how to delete it, and not run into Problems with the special Characters? What i tried - on ...
0
votes
1answer
135 views

Secure Elasticsearch with Apache 2.4

I have an Elasticsearch 1.7.5 instance running on a Ubuntu 16.04 server that I need to test after opening it up to the internet. I know you can use Mod Rewrite on Apache to lock down HTTP request ...
7
votes
4answers
23k views

Error Could not contact Elasticsearch at http://localhost:9200. - kibana dashboard

'Error Could not contact Elasticsearch at http://localhost:9200. Please ensure that Elasticsearch is reachable from your system.' Currently I am following http://packetbeat.com/getstarted I ...
1
vote
1answer
410 views

Microsoft Azure's SSD are too slow

We are running ElasticSearch on several Azure D12v2 instances, on Centos7. While indexing data, machines' IO seems to be quite poor, dancing between 3MB and 15MB per second, which is blatantly slow ...
0
votes
1answer
1k views

cloudformation circular dependancy with elasticsearch

I'm trying to create a cloudformation template that will have, among other things, an ec2 instance with and EIP and an elasticsearch domain. The issue is that I'm creating a circular dependancy that ...
3
votes
2answers
8k views

No route to host elasticsearch exception

When turning on elasticsearch, I am getting the following configuration error: [2015-04-21 20:49:45,635][INFO ][discovery.zen ] [Blackwulf] failed to send join request to master [[...
0
votes
1answer
239 views

elastic search repo creation

I have configured elasticsearch with cluster and I have on nfs shared folder that mounted but when I try to create and repo path with this location I have faced some issues with permission error": {...
2
votes
0answers
554 views

Can't access ElasticSearch through local port forward

I'm stuck at connecting to my SearchGuard-proteced ElasticSearch instance inside the company network from my local machine at home. I'm using a local SSH forward same as I got working for a phpMyAdmin ...