Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

1
vote
0answers
14 views

What is causing the Elasticsearch service to ignore my jvm.options?

In my /etc/elasticsearch/jvm.options I have set my heap size with: -Xms256m -Xmx256m However when I restart the service, memory keeps rising until it runs out and the service is stopped abruptly: [...
0
votes
0answers
9 views

logstash not able to upload data to elasticsearch even the pipeline started

I am using elasticsearch7.1.1 and logstash7.1.1 . I am trying to uplog a log file to elastic search using grok filter. pipeline is getting started but data is not getting uploaded. here is my config ...
0
votes
0answers
6 views

Read-only Access to Kibana/ESAAS

Allowing "GET" Operations only via a group is breaking kibana access. When user accesses kibana with credentials; the "discover" page is not displayed. The page "Localhost:5601/app/kibana#/discover?_g"...
0
votes
1answer
3k views

Auto delete elasticsearch data older than 30 days

I have setup a ELK stack to collect logs at central server. It is working perfectly. But by default it is holding elasticsearch index/data permanently. We just want to maintain the data for 30Days. ...
0
votes
0answers
33 views

SonarQube 7.7 Does not start on CentOS Linux

Title says everything. Same error happens with SonarQube 6.7.7 LTS but I write only 7.7 issues here. I have no version restriction which I should use. I downloaded SonarQube 7.7 from your website. ...
0
votes
2answers
597 views

haproxy redirect doesn't work for kibana

My haproxy config looks like and redirect to 5601 for kibana doesn't work. Why? ########################################################################## defaults mode http log ...
0
votes
0answers
14 views

Elasticsearch offline-archive and restore

How Elasticsearch can use 'archive'. On archive I mean logs stored offline after a specified retention time.  Is it possible to achive? How can it be done? How the offline indexes can be restored ...
0
votes
0answers
25 views

Elasticsearch connection refused

I have just installed ElasticSearch 7.1.1 on Debian 9 throw apt-get repository VPS 4GB ram .. 1vcpu service elasticsearch status elasticsearch.service - Elasticsearch Loaded: loaded (/usr/lib/...
0
votes
1answer
3k views

elasticsearch: max file descriptors [1024] for elasticsearch process is too low, increase to at least [65536]

When I tried to run the logging aggregation I found out the following error generated by elasticsearch: [2018-02-04T13:44:04,259][INFO ][o.e.b.BootstrapChecks ] [elasticsearch-logging-0] bound or ...
0
votes
1answer
23 views

Using Elasticsearch or Logstash output?

I have been going through a few tutorials on using beats to send data to elasticsearch. I noticed that some tutorials prefer to use logstash as the output which then outputs to elasticsearch. Some ...
0
votes
1answer
11 views

Elasticsearch and Kibana 7 on same host, Kibana service won't run

I am trying to install elasticsearch and kibana on a single Ubuntu 18 vm per the docs. Elasticsearch Installation and Kibana Installation NOTE: - I set the ES_HEAP_SIZE env var to half the memory of ...
0
votes
1answer
1k views

Error while setting up grafana with graphite and nginx

While trying to setup grafana with graphite using nginx server. I am getting this error: No 'Access-Control-Allow-Origin' header is present on the requested resource. I tried changing configuration ...
0
votes
1answer
2k views

Kibana @timestamp mapping & filter

I'm using following system/package: $ cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) $ rpm -q filebeat filebeat-1.3.0-1.x86_64 $ with /etc/filebeat/filebeat.yml: $ cat /etc/...
1
vote
1answer
169 views

Why SSL does not work between my 2 servers?

I have 2 servers with Ubuntu 18.04: monitoring.example.com (with ELK on a single server) www.example.com (with Filebeat) on the server ELK Create directories to store SSL certificates $ sudo mkdir ...
0
votes
0answers
26 views

Elasticsearch does not start on Ubuntu 14.04, possible error with lucene dependency

I have tried installing the elasticsearch package using apt on Ubuntu 14.04, but when I run sudo service elasticsearch status, I get the following output: * elasticsearch is not running I tried ...
-2
votes
1answer
341 views

ElasticSearch doesn't start after upgrade from 2.3.1 to 5.6.4

I was running on elasticsearch 2.3.1, but I had to upgrade to 5.6.4 (because I wanted to install Kibana/Sense), but ElasticSearch won't start anymore. This is log from ElasticSearch: java.lang....
0
votes
1answer
17 views

The kibana old repo can't be installed even after removing the previously installed newer version in centos 7

I have installed the whole elk stack 7.0 in a centos 7.0 server. I followed the following guidelines to install the kibana https://www.elastic.co/guide/en/kibana/current/rpm.html But for a testing ...
1
vote
1answer
2k views

cannot validate certificate - doesn't contain any IP SAN

I am currently in the process of installing ELK ( ElastricSearch, LogStash & Kibana) stack. My ELK server IP address is 172.29.225.32. Elastic Search config is :: # -----------------------------...
0
votes
2answers
2k views

What are records in 'netstat' command are when using logstash?

I use logstash with embedded elasticsearch. My logstash config looks like the following: output { tdout { debug => true debug_format => "json"} elasticsearch { embedded => true ...
1
vote
0answers
23 views

AWS : Replicate Amazon Elastic search and RDS

We are working on an AWS system where we are using Amazon RDS for database purposes and ElasticSearch for indexing. We have a requirement that we require weekly backup of both the services along with ...
1
vote
1answer
651 views

Elasticsearch Curator unable to connect with AWS ElasticSearch service

From a server on AWS, I push data into AWS ES service. This data appears on Kibana just fine. I am also able to perform cleanup via cURL commands from this machine. However, no machine of mine is able ...
0
votes
1answer
952 views

How to provision dashboards into kibana?

I am giving my developers the gift of kibana using the elasticsearch that is bundled with logstash. They will have kibana in their vagrant boxes. Goal: Install and configure kibana dashboards via ...
5
votes
2answers
3k views

“Index Patterns: Please specify a default index pattern” in Kibana

I'm trying to create a simple hello world for ELK and be able to see kibana reports via the internet. I've installed kibana, logstash, nginx and elastic search. Here's my /etc/logstash/conf.d/10-...
0
votes
1answer
4k views

Elastic Search Unassigned Shards

my cluster state was green and after restart of service one shard remains UNASSIGNED and status goes to yellow. I have 2 machines with 5 shard and 1 replica set settings. I am using default config ...
1
vote
1answer
1k views

Can't see pod logs in Stackdriver UI for cluster deployed on GKE

I deployed an Elasticsearch cluster on Google Kubernetes Engine. I am able to view logs of specific pods with the 'kubectl logs' command from my PC. >kubectl logs es-data-0 ... [2018-06-18T07:35:...
1
vote
1answer
991 views

iptables to allow input and output traffic to and from web server only

I have an Elastic Search server which seems to have been exploited (it's being used for a DDoS attack having had NO firewall for about a month). As a temporary measure while I create a new one I was ...
4
votes
2answers
2k views

Elasticsearch performance tuning

In a Single Node Elastic Search along with logstash, We tested with 20mb and 200mb file parsing to Elastic Search on Different types of the AWS instance i.e Medium, Large and Xlarge. Logstash conf ...
-1
votes
2answers
454 views

rsyslog with elasticsearch and forwarding custom logs

I have configured rsyslogd Log Server With Elasticsearch and Kibana. I am able to forward the /var/log/messages from the client Linux servers (CentOS 7, RHEL 6) to the central Linux Log server (CentOS ...
1
vote
1answer
1k views

vm.max_map_count problems on GKE ElasticSearch StatefulSet

A problem appeared on working ElasticSearch cluster on GKE. Nodes with "data" roles began to crash unexpectedly with an error: max virtual memory areas vm.max_map_count [65530] is too low, increase ...
1
vote
1answer
3k views

Occasional “failed to respond” from AWS hosted Elasticsearch

We have an Elasticsearch cluster hosted on Amazon Elasticsearch Service (AWS). We are using the Jest Java HTTP Rest client for ElasticSearch. Every now and then (perhaps 1 in 10,000 requests), it ...
0
votes
1answer
193 views

Elasticsearch - Kubernetes [handshake failed, mismatched cluster name]

When setting up Elasticsearch cluster on Kubernetes with 2x data, 2x master and 2x client nodes (each container on sepearete physical node) i get the following error in client-node: [2019-01-28T12:25:...
1
vote
1answer
92 views

Connect MongoDB to ElasticSearch by MongoConnector in Kubernetes

I have a MongoDB and ElasticSearch cluster which I've run by Kubernetes. I am trying to connect my ElasticSearch to MongoDB on Kubernetes but I can't find any relevant document. Can anyone help me ...
0
votes
0answers
19 views

Elastic search monitoring with graphite grafana (Better solution?)

We have created a grafana dashboard which is configured for Elasticsearch resource monitoring and we are using python script to fetch resource information from ES and put it in to Graphite. We are ...
0
votes
0answers
14 views

Elasticsearch snapshot to azure - option not appearing

I've installed the Azure repository plugin for 6.7, and configured a blob for it to write to. The directions at https://www.elastic.co/guide/en/elasticsearch/plugins/6.7/repository-azure.html have ...
1
vote
1answer
27 views

Cannot connect to open port in remote server

On my remote server, I installed ElasticSearch, which can be accessed at http://localhost:9200/: I defined an inbound security rule, Port9200, in the firewall for my remote server: As you can see, ...
1
vote
1answer
1k views

Signing/Generating SSL Certificates for ElasticSearch (search-guard)

I am trying to build my own ElasticSearch cluster and I decided to use an open source plugin called Search Guard. It requires generation of client and host/node certificates. In its demo there is a ...
2
votes
0answers
379 views

Parsing JSON event in Logstash

I have log in following format, it is a plain json with nested fields. { "level": "info", "message": { "req": { "headers": { "host": "localhost:8080", ...
9
votes
1answer
422 views

Elasticsearch dies when Logstash attempts to write data

I've got a Raspberry Pi 2 (latest Raspbian as of Apr 2015) setup that last week was running both ElasticSearch and Logstash on a test network (not a straightforward setup, but it was stable for over a ...
3
votes
3answers
2k views

kibana: no indices error

None of the existing answers helped, so here a new question. Usecase: Redirecting syslog (or) monitoring static file. I have successfully installed logstash (1.4.2), elasticsearch(1.1.1) and kibana(...
0
votes
0answers
105 views

Getting Apache2 combined logs into the Elastic Stack and using dashboards

I am looking into the ELK stack, and I mostly got it running. My goal would be to get the Apache2 dashboards running in Kibana, but I am unable to. This is how far I got: **On the Apache side:* I am ...
1
vote
0answers
52 views

ElasticSearch not starting properly

On startup, our ElasticSearch server throws aJava exception I can see in the elasticsearch.log: [2019-02-27T15:51:59,908][WARN ][o.e.m.j.JvmGcMonitorService] [KibjtLy] [gc][young][263][19] duration [...
1
vote
1answer
376 views

How to increase disk size in a stateful set

I'm managing an Elasticsearch deployment in Kubernetes. I see that the disk storage is getting close to full, so I would like to increase the size of the persistent volumes. I want to change this ...
0
votes
0answers
35 views

Moving data to a new elasticsearch index with more shards

We seem to be running out of shards on some of our larger indices on Elasticsearch 1.7. In order to utilize larger clusters with more nodes, say 20 instead of 10, I'm hoping that we can create new ...
0
votes
1answer
1k views

Why doesn't logstash grab or index the files from the mapped drive?

I don't understand why logstash is so finicky with network resources. I shared a folder on another machine and then mapped it as Z: under Windows Explorer. I've verified the path and everything. I can ...
0
votes
3answers
2k views

nginx serve Elasticsearch and Kibana on the same host

Totally new to nginx, I need a conf file to make nginx act as a reverse proxy to serve request on the same host to elasticsearch and kibana with different url path. I mean, I want: localhost/es -> ...
0
votes
0answers
170 views

Logstash: Failed to decode CEF payload

I am parsing via the cef input plugin in Logstash log files in cef format to elasticsearch. At some log files I recevice the message Failed to decode CEF payload. It seems that the problem could be ...
0
votes
1answer
99 views

Why we need to go for elasticsearch cluster with kubernetes?

I have installed kubernetes on my DC and all is working fine. I have tested several services like mongodb and web services. But my goal is to find a good solution for Elasticsearch big data. From ...
1
vote
0answers
34 views

VPC subnet error when deploying Bitname Elasticsearch to non-default GCE network

Deploying Bitnami's single-VM Elasticsearch. The error is: Invalid value for field 'resource.networkInterfaces[0]': ''. Subnetwork should be specified for custom subnetmode network It works fine if ...
0
votes
0answers
40 views

Filebeat is not sending data to Logstash

I’m trying to send alerts from Snort IDS to Elasticsearch, therefore I'm using 3 technologies: Elasticsearch- https://pastebin.com/uCNMaZFJ Logstash-https://pastebin.com/zgnbbw9K Filebeat- https://...
0
votes
0answers
149 views

fluentd error parsing auth.log

I have fluentd setup for Kubernetes which is working just fine but when add more sources it fails. I'm trying to get auth.log like this: <source> @id authlog @type tail format none ...