Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

elastic search behind haproxy

There are 4 elastic search nodes in a cluster and these 4 nodes are setup behind haproxy. Each of the node in the cluster is receiving around 900 to 1000 tcp connections, but one of the node has very ...
0
votes
0answers
11 views

APM PHP Agent Setup

Im having a really hard time setting up my APM PHP Agent on a test EC2 server, can anyone please show me how to configure it properly to send log data to my ELK Server. I tried asking this question on ...
0
votes
0answers
49 views

How to find URL for azure active directory to integrate with elasticsearch xpack?

I want to integrate my default active directory in azure with my elasticsearch server. For integration azure active directory domain URL is required. I tried by specifying the domain in elasticsearch ...
1
vote
0answers
19 views

init: logstash main process (19281) terminated with status 1

I run Amazon Linux instance with logstash 6.4.0. [root@****** dragan]# cat /etc/issue Amazon Linux AMI release 2017.03 I wanted to upgrade logstash so I uninstalled 6.4.0 and tried to install 7.3.2. ...
1
vote
0answers
17 views

Is it safe to allow POST access to the multi search API of ElasticSearch

We are running an application using elasticsearch hosted on AWS. The client-side interface uses "ReactiveSearch" for vuejs, which makes POST requests against the "_msearch" interface described here: ...
0
votes
0answers
30 views

ELK Cluster Minimum Node Count

Can we build ELK Cluster with one master and one data node, 2 nodes cluster in total? I need replication of shards. Just in case one of the node is down, does master also keep a copy of shards or is ...
0
votes
0answers
16 views

logstash not proccessing filebeat data

I have configured elk stack on a aws server and also configured filebeat on two different aws server with same config. Telnet from both filebeat server to elk server on port 5044 working fine but ...
0
votes
1answer
28 views

Elassandra “action_request_validation_exception”

Elasticsearch DEBUG: 2019-09-23T09:23:39Z starting request { "method": "HEAD", "castExists": true, "path": "/my_keyspace", "query": {} } Elasticsearch TRACE: 2019-09-23T09:23:39Z ...
1
vote
0answers
24 views

Elasticsearch is pointing to only one node on the server

I am using Elasticsearch 6.4.0 and it's working fine, except that there are 3 nodes on the server but my service points only to one node. This causes a slowdown in performance. How can I point my ...
0
votes
1answer
71 views

Unable to access ElasticSearch on AWS EC2 with private/public ip

I have an Elastic Search running on 9200 on rhel 6. I cannot reach to it in aws with private/public ip. On the other hand, i try it with curl localhost:9200, it is good. I can also ping google. I ...
2
votes
0answers
130 views

Logstash to aggregate logs into ElasticSearch

I am trying to aggregate linux logs using rsyslog into Logstash/ElasticSearch running in EKS. filebeat is already running in EKS to aggregate Kubernetes container logs. I have configured rsyslog ...
0
votes
0answers
20 views

AWS, Nginx : Redirect data from a ElasticSearch domain to nginx domain

We have our app-code running on AWS, inside a non-default VPC. All the logs are redirected to an ElasticSearch instance. There is a Kibana URL available, but it's only accessible if you do local port-...
0
votes
0answers
151 views

How to enable elasticsearch 6.8 deprecation logging? And where can I find it?

Ultimately, I want to upgrade a bunch of Python code that uses Elasticsearch 6.8 to instead use Elasticsearch 7.3. Toward that goal, I want to enable Elasticsearch 6.8 "deprecation logging". It ...
0
votes
3answers
40 views

t2.micro elasticsearch7.3 not working after editing config file

elasticsearch.service - Elasticsearch Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendo Active: failed (Result: exit-code) since Tue 2019-08-27 02:13:03 UTC; 3min 47 ...
0
votes
0answers
15 views

rsyslog: “No local error logger defined”

I'm using rsyslog to parse logs from haproxy and send them to elasticsearch. Some logs will fail to be sent to elasticsearch for various reasons. We want to keep those logs somewhere. As suggested by ...
0
votes
0answers
14 views

Filebeat custom log files / how to disable default module field mappings

I'm using filebeat to read log files that are not supported out of the box, for elasticsearch indexing. The thing is that I get 1000+ field mappings that appear to be coming from default filebeat ...
0
votes
1answer
48 views

rsyslog: Specify `action` parameters from environment variables

I'm using rsyslog to send logs to elasticsearch. It all works well in my local environment, but now I'm trying to make it more generic and inject environment variables where needed. As part of my ...
1
vote
1answer
26 views

Documents from Elasticsearch Cluster appearing twice in Grafana

I set up an Elasticsearch cluster with one dedicated master node, two master-eligible data nodes and one coordinating node. The number of replicas is set to one. There are two pipelines in Logstash, ...
0
votes
1answer
26 views

regular timeout error at certain time period

My system sends the data to Redis via web server and ultimately stacks it in Elasticsearch. The thing is that Redis connection timeout error occurs when the traffic is small. The error always occurs ...
0
votes
0answers
28 views

How do i send all logs from docker swarm to Elastic?

I have number of Linux server that have docker installed on them, all of the server are in a docker swarm, on each server i have a custom application. I also have ELK setup in AWS. I want to collect ...
0
votes
0answers
113 views

Provided Grok expressions do not match field value

edit: read below for update to initial question I'm getting Provided Grok expressions do not match field value even though _simulate works with exact same string. Error message: "message": "172.18....
0
votes
1answer
139 views

Error from server (Timeout): error when creating “STDIN”: Timeout: request did not complete within requested timeout 30s

I'm trying to Deploy the Elasticsearch cluster, yet running into following error (timeout): $ cat <<EOF | kubectl apply -f - > apiVersion: elasticsearch.k8s.elastic.co/v1alpha1 > kind: ...
1
vote
0answers
24 views
+50

Elasticsearch: how to “rescue” the documents that could not be parsed by mapping?

We're using ElasticSearch to store and inspect logs from our infrastructure. Some of those logs are required by law, and we can't afford to lose any. Unfortunately we've been parsing logs for quite ...
0
votes
1answer
183 views

How to fix unable to connect to ldapserver - Authentication Finally Failed

I'm using openldap on opendistro for elasticsearch with docker I get this error: elasticsearch | [2019-07-31T12:48:42,590][WARN ][c.a.o.s.a.BackendRegistry] [28da1860f0c0] Authentication ...
0
votes
1answer
35 views

logstash and rsyslog sends wrong hostname

I'm setting up a centralised logging system, comprising two VM's. Logger, which acts as the central syslog collector (using rsyslog) and Logstash processor; and Searcher, which acts as the ...
0
votes
1answer
60 views

Elasticsearch 404 attempting to create index through nginx proxy

I am attempting to proxy Elasticsearch through nginx. Most things seem to work, but one thing that is having problems is creating indexes. This works when connecting directly to ES: curl -XPUT http:...
3
votes
1answer
536 views

Bootstrapping a 3 node Elastic Search cluster using docker containers

I'm trying to set up a test Elastic Search cluster on 3 separate hosts, using the official 7.2.0 docker image Each container is is configured with an elasticsearch.yml which looks like this cluster....
2
votes
2answers
911 views

How to install Elasticsearch plugins with docker container?

I am a newbie in Docker, I want to install plugins in my container of elasticsearch, in this case they are: analysis-icu analysis-phonetic I know that in a traditional way it would be like this: /...
0
votes
0answers
61 views

Creating email alerts for kibana filters via sentil

I have a elk stack that works properly and can be monitored via the kibana dashboard. But as an improvement to the whole setup I have being researching for a reporting tool to setup with kibana. My ...
0
votes
0answers
15 views

Determine running version of elasticsearch, without using http interface

If an elasticsearch node supports the http interface, you can check the running version via curl -XGET 'localhost:9200' (or whatever port the http interface is on). If you want to know the installed ...
1
vote
0answers
85 views

What is causing the Elasticsearch service to ignore my jvm.options?

In my /etc/elasticsearch/jvm.options I have set my heap size with: -Xms256m -Xmx256m However when I restart the service, memory keeps rising until it runs out and the service is stopped abruptly: [...
0
votes
0answers
98 views

logstash not able to upload data to elasticsearch even the pipeline started

I am using elasticsearch 7.1.1 and logstash 7.1.1. I am trying to upload a log file to elastic search using grok filter. Pipeline is getting started, but data is not getting uploaded. Here is my ...
0
votes
0answers
18 views

Read-only Access to Kibana/ESAAS

Allowing "GET" Operations only via a group is breaking kibana access. When user accesses kibana with credentials; the "discover" page is not displayed. The page "Localhost:5601/app/kibana#/discover?_g"...
1
vote
0answers
427 views

SonarQube 7.7 Does not start on CentOS Linux

Title says everything. Same error happens with SonarQube 6.7.7 LTS but I write only 7.7 issues here. I have no version restriction which I should use. I downloaded SonarQube 7.7 from your website. ...
0
votes
0answers
73 views

Elasticsearch offline-archive and restore

How Elasticsearch can use 'archive'. On archive I mean logs stored offline after a specified retention time.  Is it possible to achive? How can it be done? How the offline indexes can be restored ...
0
votes
1answer
353 views

Elasticsearch connection refused

I have just installed ElasticSearch 7.1.1 on Debian 9 throw apt-get repository VPS 4GB ram .. 1vcpu service elasticsearch status elasticsearch.service - Elasticsearch Loaded: loaded (/usr/lib/...
0
votes
1answer
32 views

Elasticsearch and Kibana 7 on same host, Kibana service won't run

I am trying to install elasticsearch and kibana on a single Ubuntu 18 vm per the docs. Elasticsearch Installation and Kibana Installation NOTE: - I set the ES_HEAP_SIZE env var to half the memory of ...
0
votes
1answer
32 views

Using Elasticsearch or Logstash output?

I have been going through a few tutorials on using beats to send data to elasticsearch. I noticed that some tutorials prefer to use logstash as the output which then outputs to elasticsearch. Some ...
0
votes
0answers
62 views

Elasticsearch does not start on Ubuntu 14.04, possible error with lucene dependency

I have tried installing the elasticsearch package using apt on Ubuntu 14.04, but when I run sudo service elasticsearch status, I get the following output: * elasticsearch is not running I tried ...
0
votes
1answer
18 views

The kibana old repo can't be installed even after removing the previously installed newer version in centos 7

I have installed the whole elk stack 7.0 in a centos 7.0 server. I followed the following guidelines to install the kibana https://www.elastic.co/guide/en/kibana/current/rpm.html But for a testing ...
1
vote
1answer
238 views

Why SSL does not work between my 2 servers?

I have 2 servers with Ubuntu 18.04: monitoring.example.com (with ELK on a single server) www.example.com (with Filebeat) on the server ELK Create directories to store SSL certificates $ sudo mkdir ...
1
vote
0answers
23 views

AWS : Replicate Amazon Elastic search and RDS

We are working on an AWS system where we are using Amazon RDS for database purposes and ElasticSearch for indexing. We have a requirement that we require weekly backup of both the services along with ...
0
votes
0answers
23 views

Elastic search monitoring with graphite grafana (Better solution?)

We have created a grafana dashboard which is configured for Elasticsearch resource monitoring and we are using python script to fetch resource information from ES and put it in to Graphite. We are ...
0
votes
0answers
23 views

Elasticsearch snapshot to azure - option not appearing

I've installed the Azure repository plugin for 6.7, and configured a blob for it to write to. The directions at https://www.elastic.co/guide/en/elasticsearch/plugins/6.7/repository-azure.html have ...
1
vote
1answer
311 views

Cannot connect to open port in remote server

On my remote server, I installed ElasticSearch, which can be accessed at http://localhost:9200/: I defined an inbound security rule, Port9200, in the firewall for my remote server: As you can see, ...
1
vote
1answer
211 views

Connect MongoDB to ElasticSearch by MongoConnector in Kubernetes

I have a MongoDB and ElasticSearch cluster which I've run by Kubernetes. I am trying to connect my ElasticSearch to MongoDB on Kubernetes but I can't find any relevant document. Can anyone help me ...
2
votes
0answers
1k views

Parsing JSON event in Logstash

I have log in following format, it is a plain json with nested fields. { "level": "info", "message": { "req": { "headers": { "host": "localhost:8080", ...
0
votes
0answers
190 views

Getting Apache2 combined logs into the Elastic Stack and using dashboards

I am looking into the ELK stack, and I mostly got it running. My goal would be to get the Apache2 dashboards running in Kibana, but I am unable to. This is how far I got: **On the Apache side:* I am ...
1
vote
0answers
151 views

ElasticSearch not starting properly

On startup, our ElasticSearch server throws aJava exception I can see in the elasticsearch.log: [2019-02-27T15:51:59,908][WARN ][o.e.m.j.JvmGcMonitorService] [KibjtLy] [gc][young][263][19] duration [...
3
votes
2answers
1k views

How to increase disk size in a stateful set

I'm managing an Elasticsearch deployment in Kubernetes. I see that the disk storage is getting close to full, so I would like to increase the size of the persistent volumes. I want to change this ...