Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

1
vote
1answer
152 views

Why SSL does not work between my 2 servers?

I have 2 servers with Ubuntu 18.04: monitoring.example.com (with ELK on a single server) www.example.com (with Filebeat) on the server ELK Create directories to store SSL certificates $ sudo mkdir ...
0
votes
0answers
8 views

Elasticsearch install localhost:9200 can't connect, and no log files [on hold]

Nevermind L and K, E is not functioning. So what gives? What does it take to get a functioning elasticsearch install? ============= Installation hardware: Raspberry Pi 3+ Raspbian GNU/Linux 9 (...
0
votes
0answers
18 views

Elasticsearch does not start on Ubuntu 14.04, possible error with lucene dependency

I have tried installing the elasticsearch package using apt on Ubuntu 14.04, but when I run sudo service elasticsearch status, I get the following output: * elasticsearch is not running I tried ...
-2
votes
1answer
325 views

ElasticSearch doesn't start after upgrade from 2.3.1 to 5.6.4

I was running on elasticsearch 2.3.1, but I had to upgrade to 5.6.4 (because I wanted to install Kibana/Sense), but ElasticSearch won't start anymore. This is log from ElasticSearch: java.lang....
0
votes
1answer
13 views

The kibana old repo can't be installed even after removing the previously installed newer version in centos 7

I have installed the whole elk stack 7.0 in a centos 7.0 server. I followed the following guidelines to install the kibana https://www.elastic.co/guide/en/kibana/current/rpm.html But for a testing ...
1
vote
1answer
2k views

cannot validate certificate - doesn't contain any IP SAN

I am currently in the process of installing ELK ( ElastricSearch, LogStash & Kibana) stack. My ELK server IP address is 172.29.225.32. Elastic Search config is :: # -----------------------------...
0
votes
2answers
2k views

What are records in 'netstat' command are when using logstash?

I use logstash with embedded elasticsearch. My logstash config looks like the following: output { tdout { debug => true debug_format => "json"} elasticsearch { embedded => true ...
1
vote
0answers
19 views

AWS : Replicate Amazon Elastic search and RDS

We are working on an AWS system where we are using Amazon RDS for database purposes and ElasticSearch for indexing. We have a requirement that we require weekly backup of both the services along with ...
1
vote
1answer
634 views

Elasticsearch Curator unable to connect with AWS ElasticSearch service

From a server on AWS, I push data into AWS ES service. This data appears on Kibana just fine. I am also able to perform cleanup via cURL commands from this machine. However, no machine of mine is able ...
0
votes
1answer
935 views

How to provision dashboards into kibana?

I am giving my developers the gift of kibana using the elasticsearch that is bundled with logstash. They will have kibana in their vagrant boxes. Goal: Install and configure kibana dashboards via ...
5
votes
2answers
3k views

“Index Patterns: Please specify a default index pattern” in Kibana

I'm trying to create a simple hello world for ELK and be able to see kibana reports via the internet. I've installed kibana, logstash, nginx and elastic search. Here's my /etc/logstash/conf.d/10-...
0
votes
1answer
4k views

Elastic Search Unassigned Shards

my cluster state was green and after restart of service one shard remains UNASSIGNED and status goes to yellow. I have 2 machines with 5 shard and 1 replica set settings. I am using default config ...
1
vote
1answer
1k views

Can't see pod logs in Stackdriver UI for cluster deployed on GKE

I deployed an Elasticsearch cluster on Google Kubernetes Engine. I am able to view logs of specific pods with the 'kubectl logs' command from my PC. >kubectl logs es-data-0 ... [2018-06-18T07:35:...
1
vote
1answer
975 views

iptables to allow input and output traffic to and from web server only

I have an Elastic Search server which seems to have been exploited (it's being used for a DDoS attack having had NO firewall for about a month). As a temporary measure while I create a new one I was ...
4
votes
2answers
2k views

Elasticsearch performance tuning

In a Single Node Elastic Search along with logstash, We tested with 20mb and 200mb file parsing to Elastic Search on Different types of the AWS instance i.e Medium, Large and Xlarge. Logstash conf ...
-1
votes
2answers
444 views

rsyslog with elasticsearch and forwarding custom logs

I have configured rsyslogd Log Server With Elasticsearch and Kibana. I am able to forward the /var/log/messages from the client Linux servers (CentOS 7, RHEL 6) to the central Linux Log server (CentOS ...
1
vote
1answer
1k views

vm.max_map_count problems on GKE ElasticSearch StatefulSet

A problem appeared on working ElasticSearch cluster on GKE. Nodes with "data" roles began to crash unexpectedly with an error: max virtual memory areas vm.max_map_count [65530] is too low, increase ...
1
vote
1answer
2k views

Occasional “failed to respond” from AWS hosted Elasticsearch

We have an Elasticsearch cluster hosted on Amazon Elasticsearch Service (AWS). We are using the Jest Java HTTP Rest client for ElasticSearch. Every now and then (perhaps 1 in 10,000 requests), it ...
0
votes
1answer
150 views

Elasticsearch - Kubernetes [handshake failed, mismatched cluster name]

When setting up Elasticsearch cluster on Kubernetes with 2x data, 2x master and 2x client nodes (each container on sepearete physical node) i get the following error in client-node: [2019-01-28T12:25:...
1
vote
1answer
76 views

Connect MongoDB to ElasticSearch by MongoConnector in Kubernetes

I have a MongoDB and ElasticSearch cluster which I've run by Kubernetes. I am trying to connect my ElasticSearch to MongoDB on Kubernetes but I can't find any relevant document. Can anyone help me ...
0
votes
0answers
18 views

Elastic search monitoring with graphite grafana (Better solution?)

We have created a grafana dashboard which is configured for Elasticsearch resource monitoring and we are using python script to fetch resource information from ES and put it in to Graphite. We are ...
0
votes
0answers
12 views

Elasticsearch snapshot to azure - option not appearing

I've installed the Azure repository plugin for 6.7, and configured a blob for it to write to. The directions at https://www.elastic.co/guide/en/elasticsearch/plugins/6.7/repository-azure.html have ...
1
vote
1answer
26 views

Cannot connect to open port in remote server

On my remote server, I installed ElasticSearch, which can be accessed at http://localhost:9200/: I defined an inbound security rule, Port9200, in the firewall for my remote server: As you can see, ...
1
vote
1answer
1k views

Signing/Generating SSL Certificates for ElasticSearch (search-guard)

I am trying to build my own ElasticSearch cluster and I decided to use an open source plugin called Search Guard. It requires generation of client and host/node certificates. In its demo there is a ...
2
votes
0answers
223 views

Parsing JSON event in Logstash

I have log in following format, it is a plain json with nested fields. { "level": "info", "message": { "req": { "headers": { "host": "localhost:8080", ...
9
votes
1answer
421 views

Elasticsearch dies when Logstash attempts to write data

I've got a Raspberry Pi 2 (latest Raspbian as of Apr 2015) setup that last week was running both ElasticSearch and Logstash on a test network (not a straightforward setup, but it was stable for over a ...
3
votes
3answers
2k views

kibana: no indices error

None of the existing answers helped, so here a new question. Usecase: Redirecting syslog (or) monitoring static file. I have successfully installed logstash (1.4.2), elasticsearch(1.1.1) and kibana(...
0
votes
0answers
83 views

Getting Apache2 combined logs into the Elastic Stack and using dashboards

I am looking into the ELK stack, and I mostly got it running. My goal would be to get the Apache2 dashboards running in Kibana, but I am unable to. This is how far I got: **On the Apache side:* I am ...
1
vote
0answers
43 views

ElasticSearch not starting properly

On startup, our ElasticSearch server throws aJava exception I can see in the elasticsearch.log: [2019-02-27T15:51:59,908][WARN ][o.e.m.j.JvmGcMonitorService] [KibjtLy] [gc][young][263][19] duration [...
1
vote
1answer
263 views

How to increase disk size in a stateful set

I'm managing an Elasticsearch deployment in Kubernetes. I see that the disk storage is getting close to full, so I would like to increase the size of the persistent volumes. I want to change this ...
0
votes
0answers
34 views

Moving data to a new elasticsearch index with more shards

We seem to be running out of shards on some of our larger indices on Elasticsearch 1.7. In order to utilize larger clusters with more nodes, say 20 instead of 10, I'm hoping that we can create new ...
0
votes
1answer
1k views

Why doesn't logstash grab or index the files from the mapped drive?

I don't understand why logstash is so finicky with network resources. I shared a folder on another machine and then mapped it as Z: under Windows Explorer. I've verified the path and everything. I can ...
0
votes
3answers
2k views

nginx serve Elasticsearch and Kibana on the same host

Totally new to nginx, I need a conf file to make nginx act as a reverse proxy to serve request on the same host to elasticsearch and kibana with different url path. I mean, I want: localhost/es -> ...
0
votes
0answers
122 views

Logstash: Failed to decode CEF payload

I am parsing via the cef input plugin in Logstash log files in cef format to elasticsearch. At some log files I recevice the message Failed to decode CEF payload. It seems that the problem could be ...
0
votes
1answer
95 views

Why we need to go for elasticsearch cluster with kubernetes?

I have installed kubernetes on my DC and all is working fine. I have tested several services like mongodb and web services. But my goal is to find a good solution for Elasticsearch big data. From ...
1
vote
0answers
33 views

VPC subnet error when deploying Bitname Elasticsearch to non-default GCE network

Deploying Bitnami's single-VM Elasticsearch. The error is: Invalid value for field 'resource.networkInterfaces[0]': ''. Subnetwork should be specified for custom subnetmode network It works fine if ...
0
votes
0answers
39 views

Filebeat is not sending data to Logstash

I’m trying to send alerts from Snort IDS to Elasticsearch, therefore I'm using 3 technologies: Elasticsearch- https://pastebin.com/uCNMaZFJ Logstash-https://pastebin.com/zgnbbw9K Filebeat- https://...
0
votes
0answers
117 views

fluentd error parsing auth.log

I have fluentd setup for Kubernetes which is working just fine but when add more sources it fails. I'm trying to get auth.log like this: <source> @id authlog @type tail format none ...
0
votes
2answers
132 views

Elastic Search using a lot of memory despite low heap size

We have configured Elastic Search JVM options /etc/elasticsearch/jvm.options to use a fixed heap size of -Xms512m -Xmx512m This can be confirmed as working when checking the Elastic Search process: ...
0
votes
1answer
1k views

Error while setting up grafana with graphite and nginx

While trying to setup grafana with graphite using nginx server. I am getting this error: No 'Access-Control-Allow-Origin' header is present on the requested resource. I tried changing configuration ...
0
votes
1answer
2k views

elasticsearch: max file descriptors [1024] for elasticsearch process is too low, increase to at least [65536]

When I tried to run the logging aggregation I found out the following error generated by elasticsearch: [2018-02-04T13:44:04,259][INFO ][o.e.b.BootstrapChecks ] [elasticsearch-logging-0] bound or ...
0
votes
0answers
19 views

Migrating from a public elastic cluster to VPC cluster

I currently, have one public elastic cluster with one data node(m3.xlarge.elasticsearch) which is working fine. I would like to move from the public domain to VPC based domain. I was able to take a ...
0
votes
1answer
1k views

Kibana @timestamp mapping & filter

I'm using following system/package: $ cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) $ rpm -q filebeat filebeat-1.3.0-1.x86_64 $ with /etc/filebeat/filebeat.yml: $ cat /etc/...
0
votes
0answers
136 views

Logstash isn't accepting the self signed certificates and throws a sslv3 alert handshake failure

I have set up an elsatic stack in my laptop and I have configured full stack (filebeat, logstash, elasticsearch and kibana) in it. But now I need to connect filebeat and logstash in a secure manner ...
0
votes
0answers
36 views

AWS Elasticsearch access policy with Kinesis and EC2

I am currently working on an AWS setup where data is pushed to an Elasticsearch service via Amazon Kinesis (Firehose). This data has to be accessible from all EC2 instances that are connected to a ...
2
votes
3answers
3k views

Logstash / Elasticsearch - trasform fields to lowercase in output conf

I have a standard ELK stack currently storing numerous log outputs. I'm trying to separate my indices to be source-specific. As part of my FileBeats config, some standard fields are always generated ...
1
vote
1answer
121 views

Logstash's ElasticSearch Input Plugin should be used to output to ElasticSearch

We hope to use ElasticSearch Input Plugin from Logstash to output with ElasticSearch. This feature has been successfully collected by MongoDB Input Plugin in real time. However, the ElasticSearch ...
0
votes
1answer
195 views

Logstash Input MongoDB Error “No suitable driver found for jdbc:mongo://localhost:27017/test”

logasth conf "input part" input { jdbc { jdbc_driver_library => "/home/jskang/mongo-java-driver-3.2.2.jar" jdbc_driver_class => "com.mongodb.MongoClient" jdbc_connection_string =&...
0
votes
0answers
189 views

Ceph vs GlusterFS for elasticsearch storage

Elasticsearch will be holding logs mostly, so we expect sequential read/writes. Does anyone have experience on this? Elasticsearch will run via openshift, if it makes any difference.
0
votes
1answer
99 views

Configure Kibana web page (only) to require a login

I have Elasticsearch 6.4.3 and Kibana 6.4.3 installed on Debian 8 Jessie. I've looked into X-Pack and Shield, which both seem to be mostly concerned with security within Elasticsearch, and how to ...