Questions tagged [elasticsearch]
Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.
379
questions
0
votes
0
answers
849
views
Elasticsearch systemd setup
I am configuring Elasticsearch 2.3.3 (yes, outdated) on CentOS 7.7. We were running it on CentOS 6 with InitV script. I followed this specification to create systemd script and get inspired with the ...
- 305
0
votes
1
answer
2k
views
Can't access Kibana remotely - Can curl successfully on machine
I've installed Kibana on rhel and am I'm trying to access remotely on http://server-url:5601.
I've installed elasticsearch and am able to access that remotely on http://server-url:9200
I've added
...
0
votes
0
answers
17
views
Adding context paths to ELK log analytics?
I'm looking to install the ELK stack on my application server by following the instructions here:
I can see you can then access the various ELK services on localhost via certain ports. Is there a way ...
0
votes
1
answer
943
views
Grok filter is not working properly
I have Filebeat-7.1 installed in a Debian server, this Filebeat send data from files in this Debian server to server with Logstash 7.6 , here are the files config
Filebeat.yml:
#=====================...
- 1
0
votes
2
answers
501
views
Kibana and Logstash: When is a field not a field?
I've enabled the grok filter in our logstash configuration in order to parse logs from HAProxy using the %{HAPROXYHTTP} and %{HAPROXYTCP} patterns. This seems to work great and viewing the details for ...
- 44.9k
1
vote
0
answers
552
views
Scaling Elasticsearch down to single-node
Is it possible to scale Elasticsearch from multiple nodes down to one node?
I have a 3-node cluster that is way overkill for the amount of data being logged.
To scale it down, I set "cluster.routing....
0
votes
1
answer
767
views
Does Elasticsearch have capability to form ES cluster with only one ES docker service running multiple replicas of it in Docker-Swarm
I wanna run only one elasticsearch service in docker-swarm having multiple replicas of this service.
Does Elasticsearch have capability to form ES cluster with only one ES docker service running ...
2
votes
0
answers
140
views
Index rollover latency
I have created an ILM policy as follows,
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_docs": 30
},
...
- 151
0
votes
1
answer
158
views
Do we need a back up of Elastic Search data in a database in case of failure, or is there a solid permanant storage solution with Elastic Search?
We are looking to use Elastic Search in a major software system, which needs financial logs and records kept for many years. We are using Elastic Search, but are a little concerned whether Elastic is ...
- 123
1
vote
1
answer
741
views
Virtual machine disk I/O latency spikes
Since past few days I have strange I/O spikes in one virtual machine.
Its
2.6.32-504.el6.x86_64 #1 SMP Tue Sep 16 01:56:35 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server ...
0
votes
1
answer
248
views
Elasticsearch - how to disable client access on data node?
I am fairly new to Elasticsearch and was watching some tutorials that are somewhat outdated.
In the tutorial, they said that if one has dedicated data nodes, it is recommended to disable client ...
- 495
0
votes
1
answer
252
views
Integrating nginx logs and elasticsearch app-search
I'm trying to setup a self-managed docker appsearch instance, together with kibana and elasticsearch, queried by a uvicorn python app, proxied by a nginx webserver
My current issue is that the ...
- 121
0
votes
1
answer
185
views
Pi4+ runs Elasticsearch cluster of 1, gives yellow status
I am currently running Elasticsearch 7.3.2 on a Rasbian (Buster) instance on a Raspberry Pi 4+. It looks like it was running in a green status for a few days, processing files, but suddenly, i ...
- 101
0
votes
1
answer
628
views
AWS ElasticSearch service does not provide Kibana users management
First I'm going to explain my current architecture. I have an EC2 instance running with a Logstash daemon, as input an http configuration and as output my AWS ElasticSearch instance.
As I said before,...
- 101
0
votes
0
answers
2k
views
ELK - Logstash not picking up syslog events
I'm setting up a ELK cluster using Centos 8 and version 7.4 of Elasticsearch, Logstash and Kibana.
My issue is with Logstash not picking up the events coming through syslog.
Configuring Logstash to ...
- 267
0
votes
0
answers
1k
views
Run OCR in Nextcloud with Docker
I'm using the official Docker image nextcloud:latest to run my Nextcloud instance. I managed to install Elasticsearch (on the host) and created an index. Search works so far.
Now I want to OCR PDFs ...
- 3,157
0
votes
0
answers
1k
views
elastic search behind haproxy
There are 4 elastic search nodes in a cluster and these 4 nodes are setup behind haproxy. Each of the node in the cluster is receiving around 900 to 1000 tcp connections, but one of the node has very ...
1
vote
0
answers
162
views
init: logstash main process (19281) terminated with status 1
I run Amazon Linux instance with logstash 6.4.0.
[root@****** dragan]# cat /etc/issue
Amazon Linux AMI release 2017.03
I wanted to upgrade logstash so I uninstalled 6.4.0 and tried to install 7.3.2. ...
- 11
1
vote
0
answers
107
views
Is it safe to allow POST access to the multi search API of ElasticSearch
We are running an application using elasticsearch hosted on AWS. The client-side interface uses "ReactiveSearch" for vuejs, which makes POST requests against the "_msearch" interface described here: ...
- 148
0
votes
1
answer
833
views
Elassandra "action_request_validation_exception"
Elasticsearch DEBUG: 2019-09-23T09:23:39Z
starting request {
"method": "HEAD",
"castExists": true,
"path": "/my_keyspace",
"query": {}
}
Elasticsearch TRACE: 2019-09-23T09:23:39Z
...
- 223
1
vote
0
answers
31
views
Elasticsearch is pointing to only one node on the server
I am using Elasticsearch 6.4.0 and it's working fine, except that there are 3 nodes on the server but my service points only to one node. This causes a slowdown in performance.
How can I point my ...
Azhar
0
votes
1
answer
2k
views
Unable to access ElasticSearch on AWS EC2 with private/public ip
I have an Elastic Search running on 9200 on rhel 6. I cannot reach to it in aws with private/public ip. On the other hand, i try it with curl localhost:9200, it is good. I can also ping google.
I ...
- 107
2
votes
0
answers
342
views
Logstash to aggregate logs into ElasticSearch
I am trying to aggregate linux logs using rsyslog into Logstash/ElasticSearch running in EKS.
filebeat is already running in EKS to aggregate Kubernetes container logs.
I have configured rsyslog ...
- 81
0
votes
3
answers
119
views
t2.micro elasticsearch7.3 not working after editing config file
elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendo
Active: failed (Result: exit-code) since Tue 2019-08-27 02:13:03 UTC; 3min 47
...
- 1
1
vote
2
answers
3k
views
rsyslog: Specify `action` parameters from environment variables
I'm using rsyslog to send logs to elasticsearch. It all works well in my local environment, but now I'm trying to make it more generic and inject environment variables where needed.
As part of my ...
- 360
1
vote
1
answer
91
views
Documents from Elasticsearch Cluster appearing twice in Grafana
I set up an Elasticsearch cluster with one dedicated master node, two master-eligible data nodes and one coordinating node.
The number of replicas is set to one.
There are two pipelines in Logstash, ...
- 141
0
votes
1
answer
97
views
regular timeout error at certain time period
My system sends the data to Redis via web server and ultimately stacks it in Elasticsearch.
The thing is that Redis connection timeout error occurs when the traffic is small.
The error always occurs ...
0
votes
1
answer
1k
views
Error from server (Timeout): error when creating "STDIN": Timeout: request did not complete within requested timeout 30s
I'm trying to Deploy the Elasticsearch cluster, yet running into following error (timeout):
$ cat <<EOF | kubectl apply -f -
> apiVersion: elasticsearch.k8s.elastic.co/v1alpha1
> kind: ...
- 13.4k
2
votes
2
answers
2k
views
Elasticsearch: how to "rescue" the documents that could not be parsed by mapping?
We're using ElasticSearch to store and inspect logs from our infrastructure. Some of those logs are required by law, and we can't afford to lose any.
We've been parsing logs for quite some time ...
- 360
0
votes
1
answer
2k
views
How to fix unable to connect to ldapserver - Authentication Finally Failed
I'm using openldap on opendistro for elasticsearch with docker
I get this error:
elasticsearch | [2019-07-31T12:48:42,590][WARN ][c.a.o.s.a.BackendRegistry] [28da1860f0c0] Authentication ...
0
votes
1
answer
543
views
logstash and rsyslog sends wrong hostname
I'm setting up a centralised logging system, comprising two VM's.
Logger, which acts as the central syslog collector (using rsyslog) and Logstash processor; and
Searcher, which acts as the ...
0
votes
1
answer
383
views
Elasticsearch 404 attempting to create index through nginx proxy
I am attempting to proxy Elasticsearch through nginx. Most things seem to work, but one thing that is having problems is creating indexes.
This works when connecting directly to ES:
curl -XPUT http:...
- 489
3
votes
1
answer
2k
views
Bootstrapping a 3 node Elastic Search cluster using docker containers
I'm trying to set up a test Elastic Search cluster on 3 separate hosts, using the official 7.2.0 docker image
Each container is is configured with an elasticsearch.yml which looks like this
cluster....
- 1,576
7
votes
2
answers
22k
views
How to install Elasticsearch plugins with docker container?
I am a newbie in Docker, I want to install plugins in my container of elasticsearch, in this case they are:
analysis-icu
analysis-phonetic
I know that in a traditional way it would be like this:
/...
- 173
1
vote
0
answers
2k
views
What is causing the Elasticsearch service to ignore my jvm.options?
In my /etc/elasticsearch/jvm.options
I have set my heap size with:
-Xms256m
-Xmx256m
However when I restart the service, memory keeps rising until it runs out and the service is stopped abruptly:
[...
- 453
0
votes
1
answer
972
views
logstash not able to upload data to elasticsearch even the pipeline started
I am using elasticsearch 7.1.1 and logstash 7.1.1. I am trying to upload a log file to elastic search using grok filter.
Pipeline is getting started, but data is not getting uploaded.
Here is my ...
- 11
1
vote
0
answers
2k
views
SonarQube 7.7 Does not start on CentOS Linux
Title says everything. Same error happens with SonarQube 6.7.7 LTS but I write only 7.7 issues here. I have no version restriction which I should use.
I downloaded SonarQube 7.7 from your website. ...
- 281
0
votes
1
answer
2k
views
Elasticsearch connection refused
I have just installed ElasticSearch 7.1.1 on Debian 9 throw apt-get repository
VPS 4GB ram .. 1vcpu
service elasticsearch status
elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/...
0
votes
1
answer
437
views
Elasticsearch and Kibana 7 on same host, Kibana service won't run
I am trying to install elasticsearch and kibana on a single Ubuntu 18 vm per the docs. Elasticsearch Installation and Kibana Installation
NOTE:
- I set the ES_HEAP_SIZE env var to half the memory of ...
- 189
0
votes
1
answer
143
views
Using Elasticsearch or Logstash output?
I have been going through a few tutorials on using beats to send data to elasticsearch.
I noticed that some tutorials prefer to use logstash as the output which then outputs to elasticsearch. Some ...
- 453
0
votes
0
answers
284
views
Elasticsearch does not start on Ubuntu 14.04, possible error with lucene dependency
I have tried installing the elasticsearch package using apt on Ubuntu 14.04, but when I run sudo service elasticsearch status, I get the following output:
* elasticsearch is not running
I tried ...
- 101
0
votes
1
answer
68
views
The kibana old repo can't be installed even after removing the previously installed newer version in centos 7
I have installed the whole elk stack 7.0 in a centos 7.0 server. I followed the following guidelines to install the kibana
https://www.elastic.co/guide/en/kibana/current/rpm.html
But for a testing ...
- 13
1
vote
1
answer
1k
views
Why SSL does not work between my 2 servers?
I have 2 servers with Ubuntu 18.04:
monitoring.example.com (with ELK on a single server)
www.example.com (with Filebeat)
on the server ELK
Create directories to store SSL certificates
$ sudo mkdir ...
- 13
1
vote
0
answers
51
views
AWS : Replicate Amazon Elastic search and RDS
We are working on an AWS system where we are using Amazon RDS for database purposes and ElasticSearch for indexing. We have a requirement that we require weekly backup of both the services along with ...
- 11
1
vote
1
answer
5k
views
Cannot connect to open port in remote server
On my remote server, I installed ElasticSearch, which can be accessed at http://localhost:9200/:
I defined an inbound security rule, Port9200, in the firewall for my remote server:
As you can see, ...
- 121
1
vote
1
answer
580
views
Connect MongoDB to ElasticSearch by MongoConnector in Kubernetes
I have a MongoDB and ElasticSearch cluster which I've run by Kubernetes.
I am trying to connect my ElasticSearch to MongoDB on Kubernetes but I can't find any relevant document.
Can anyone help me ...
2
votes
0
answers
4k
views
Parsing JSON event in Logstash
I have log in following format, it is a plain json with nested fields.
{
"level": "info",
"message": {
"req": {
"headers": {
"host": "localhost:8080",
...
- 21
1
vote
0
answers
992
views
ElasticSearch not starting properly
On startup, our ElasticSearch server throws aJava exception I can see in the elasticsearch.log:
[2019-02-27T15:51:59,908][WARN ][o.e.m.j.JvmGcMonitorService] [KibjtLy] [gc][young][263][19] duration [...
- 255
37
votes
6
answers
55k
views
How to increase disk size in a stateful set
I'm managing an Elasticsearch deployment in Kubernetes. I see that the disk storage is getting close to full, so I would like to increase the size of the persistent volumes.
I want to change this ...
- 471
0
votes
1
answer
782
views
Why we need to go for elasticsearch cluster with kubernetes?
I have installed kubernetes on my DC and all is working fine. I have tested several services like mongodb and web services. But my goal is to find a good solution for Elasticsearch big data.
From ...
- 91