Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Application and Server Communcation Between Two Firewalls

I have a test scenario where I have an application elastic-agent (behind Pfsense Firewall A) that needs to send communication over 8220 and 9200 to my server elasticsearch (behind Pfsense Firewall B). ...
0 votes
1 answer
16 views

Restricting Kibana to TLSv1.2

What is the setting that needs to be set in the kibana.yml file (or other file) to restrict Kibana to TLSv1.2 over HTTP? I believe that it uses TLSv1.0, TLSv1.1, and TLSv1.2 by default. I need to ...
  • 1
0 votes
0 answers
51 views

Elasticsearch node high CPU usage

We are trying to make sense of high CPU usage on a write heavy Elasticsearch node. The node is receiving ~70 update and insert requests per second for two indexes with a max data size of 7kb, and 12 ...
  • 1
1 vote
0 answers
18 views

Suricata / Filebeat / ELK - iptables tee - Create virtual hosts

I have an IDS setup as follow: Hardware / interfaces WAN <----(brwan)> ROUTER / AP <(br0)----> LAN \ -----(eth1)>...
0 votes
1 answer
33 views

Handle 150 req/s and 2TB of logs for 7 days of retention available by API

I would like to get advise from experienced people to build a HA infrastructure to log 2To of data in JSON format every week. I need to have a retention time of 7 days and need to be able to requests ...
0 votes
1 answer
159 views

curl -X GET http://localhost:9200 return Access Denied

I am trying to install and setup Elasticsearch 8 on Centos 7 server, the service started but when I do curl -x GET HTTP://localhost:9200, or curl -x GET HTTP://127.0.0.1:9200 or curl -x GET HTTP://(...
  • 1
0 votes
0 answers
141 views

Free storage space in amazon open search data node is low and others nodes has more than enough free space

In my AWS open search i am using 8 data nodes (r5.2xlarge.search) and 3 master nodes (r5.large.search) and each data nodes has allocated to a space of 200gb each. And also added my shard value as 4....
  • 37
0 votes
1 answer
163 views

Graylog - data from newest indice not returned on search

I have a Graylog server (newest version) collecting data from nginx. It´s been running for a while. I retrieve the collected data by querying my Elasticsearch cluster (v7 newest version) which ...
1 vote
0 answers
91 views

Amazon OpenSearch Service stably or less downtime?

Can anyone point out, What is the best way for the elastic search to run stably? We frequently get messages about it requiring updates and that an update requires some downtime. Could we have 2 ...
0 votes
0 answers
401 views

how having Opensearch and openserach dashboard in same docker file

My final goal is to have opensearch and opensearch-dashboard on same unix server without docker But in first time i need to test them in uniq dockerFiles I have tried without succes FROM centos:...
  • 1
0 votes
0 answers
108 views

how to confirgure the metricbeat data collection and index rotation

i trying to configure metricbeat data collection with date wise pattern and tired to rotate the metricbeat logs and made changes in metricbeat.ym in output.elasticsearch: session the changes are ...
0 votes
0 answers
125 views

How to use "exclude_line" in filebeat yml

I'm trying to exclude a specific type if lines from my logs but not sure what regex pattern I should be using. Please advise. I want to exclude any lines that have "/ws/api/authenticate". ...
0 votes
0 answers
79 views

Evebox doesn´t connect to elasticsearch

I have an evebox showing me this error when it connects to my elasticsearch. here it is my evebox.yml and sytemctl status evebox.service outputs data-directory: /var/lib/evebox http: tls: ...
1 vote
0 answers
134 views

On Debian system, keepalived is consuming full cpu usage on both kibana nodes

I have master/slave kibana machines on Debian 10. From few days, cpu usage of keepalived daemon on both nodes are reaching to 100%. After keepalived restart it goes to normal, and this loops forever. ...
0 votes
0 answers
20 views

ElasticSearch Cluster on AKS with loadbalancer

I am trying to deploy elasticsearch to AKS with a loadbalancer. What I am struggling to achieve is to have a load balancer that only directs traffic to my client nodes. This is what I have: ...
0 votes
0 answers
26 views

How to fetch Individual output of different match_phrase inside one request

I am able to run below query which is giving response as total of two match_phrase. Inserting dummy data as below. POST /mod1/_bulk { "index" : { } } { "msg": "BA1" } { &...
  • 11
0 votes
1 answer
40 views

How to pass values to query externally

I have below similar logs. I have created dummy index and created mapping like below in dev-tools PUT new { "mappings": { "properties": { "@timestamp": { ...
  • 11
0 votes
0 answers
23 views

Elasticsearch php client : server priority

Actual configuration: Consider a cluster of Elasticsearch 6.8 servers. Each server node has a cluster of ETL nodes (PHP Elasticsearch client). All the ETLs are configured to connect to all ...
  • 133
0 votes
1 answer
32 views

Can logstash "pull" data?

I have two servers. Server A is running Elasticsearch and Logstash. Server B is running filebeat and is also the server which contains all the logs I'm trying to analyse. Server A is behind a firewall,...
  • 871
0 votes
0 answers
24 views

Enabling the elasticsearch token service without enabling TLS

I've got elasticsearch and kibana running in docker containers, on an isolated docker network. The only port that is "exposed" is Kibana's (unencrypted) web interface. And this is only ...
  • 295
0 votes
0 answers
185 views

Finding source of high IOWait

I have a 5 node elasticsearch cluster. One host has had consistently high IOWait% (40+) for several weeks. The others seem fine (<10%). Notable: Host in question is not the cluster master New ...
  • 2,327
0 votes
0 answers
264 views

Elasticsearch on docker fills up the disk quickly with no apparent reason

For quite a few months everything was working fine and this problem didn't exists. After a crash loop in one of the containers which started producing a lot of logs the server got full. That problem ...
0 votes
0 answers
254 views

Fluentd unable to parse Json logs generated from node pm2 server

I am using fluentd, Elasticsearch and Grafana stack for logging and monitoring setup. I am tailing the logs(in JSON format) coming from NodeJS container/pod deployed in EKS cluster and dumping into ...
  • 1
0 votes
1 answer
193 views

graylog 4.0 - Injector could not be created

Trying to set up graylog with official graylog2.graylog ansible role. Installed Elasticsearch (v7.10.2) cluster on three separate servers with official elasticsearch ansible role (node_1 - master, ...
  • 101
1 vote
0 answers
790 views

My elastic search is not showing any data using elastic-agent

Surprisingly my elastic version 7.15 with basic license have installed the fleet server and then configured windows-policy with elastic-endpoint and windows integration. Have enabled all the rules; ...
0 votes
1 answer
231 views

Running Enterprise Search as a service

Elastic's Enterprise Search Installation Guide goes as far as starting the process locally - which is obviously not very stable. When installed from a .deb or .rpm, a service is actually set up and ...
  • 111
1 vote
1 answer
410 views

Is there a way to use the ingest attachment plugin with Elastic App Search

I'm working on a portal that hosts multiple types of documentation (HTML, PDF, PPTx, DocX) and makes them all searchable in one place. We could achieve this using the "standard" out of the ...
  • 111
1 vote
1 answer
522 views

Kibana does not trust Let's Encrypt CA in Kubernetes Helm Setup

I have a k8s cluster configured with calico as networking system. I'm running slightly customized versions of helm elastic/elasticsearch and elastic/kibana with security enabled. For security, I use ...
  • 31
2 votes
1 answer
1k views

When to use a new index in Graylog (Elasticsearch)?

I have been searching for days now to find a good explanation on how indices are used by Graylog/Elasticsearch and when to create a new one. There is a lot of info about sharding indices but not much ...
  • 133
0 votes
1 answer
190 views

Filebeat on ECK with AWS Module Fails Due To Metadata Error

We are running an Elastic Stack with ECK in EKS (7.8). We noticed that our filebeat daemonset and the AWS module were not processing logs from S3 and our SQS queues backing up. Looking at the logs on ...
0 votes
1 answer
2k views

Elasticsearch - Can I delete a data-stream backing index?

I have a few data-streams configured in my Elasticsearch cluster (with ILM policy). Suppose I run into storage problems and etc. and I want to prematurely delete a few backing indices of the data-...
  • 405
0 votes
1 answer
5k views

Kibana cannot be accessed through port 5601 in browser [duplicate]

I cannot get to kibana through web UI. I'm set the necessary ports in firewall-cmd, I've configured kibana.yml and elasticsearch.yml. I'm not seeing any relevant errors, or at least don't know if they ...
  • 113
0 votes
0 answers
43 views

Multiple Web Application - Cache Layer Design

I have multiple client e-commerce web applications running on a single VM, each web application is a node.js express application. The express web application communicates with the back end via API's ...
  • 11
0 votes
1 answer
179 views

How to stop ElasticSearch background task

According to the CLI help of ElasticSearch, you can start ElasticSearch in the background with elasticsearch -d or elasticsearch --daemonize. This is what I use all the time locally, as it seems to ...
  • 273
0 votes
0 answers
29 views

Ubuntu server 'partially' responsive

I have an Ubuntu 18.04 server that has periodically unresponsive - I can ping it, and it appears that some https services are running (albeit v. slowly), however it hangs if I try to log in (on the ...
0 votes
1 answer
53 views

realtime monitoring of linux server network

We have a few Linux server which act as Edge network, We want realtime (or near-realtime) information on network bandwidth, top IP addresses and network bandwidth per them and other network parameters ...
  • 141
0 votes
1 answer
87 views

How to display the logs of a VM on ELK stack which is running on another VM?

-ELK is running on localhost & successfully getting logs. -Both VM's are on same server. -OS on both VM'S is Ubuntu 18-04 LTS.
0 votes
1 answer
195 views

NGINX - Access port via SSL

I have ElasticSearch and NGINX set up on my server. Accessing it via SSL is failing. I can access ElasticSearch fine via: http://ipaddress:9200 http://mydomain.co.uk:9200 The domain was then set up ...
  • 151
0 votes
0 answers
22 views

Storage balancing in elasticsearch cluster

I've a Elasticsearch cluster with 5 nodes. I've this used storage repartition: Node1: 76% Node2: 94% Node3: 88% Node4: 73% Node5: 74% How to do to storage balancing/leveling the used storage ? For ...
  • 1
1 vote
0 answers
260 views

Correct reverse proxy to aws-es-proxy instance?

I installed a VPC with an Elasticsearch cluster with kibana on AWS. As that is VPC, it is not accessible publicly. Thus, to be able to access the kibana interface, I installed the https://github.com/...
0 votes
1 answer
85 views

How to use filebeat to find password in log

I have an Elasticsearch 7.1 and i have configure filebeat to collect all log. I want to check if i have password in log. So anyone have an idea how can i find all password in log using filebeat. Thank ...
  • 21
1 vote
2 answers
2k views

Can you help me understand this ElasticSearch permissions error?

I'm attempting to make a request to an Elastic.co hosted ElasticSearch instance. I'm receiving this error: AuthorizationException(403, 'security_exception', 'action [indices:monitor/stats] is ...
  • 465
0 votes
1 answer
541 views

Portainer: Pass vm.max_map_count to SonarQube Docker container

I need to set up a SonarQube container on my Portainer instance. SonarQube uses Elasticsearch, which needs a kernel setting passed from the host. This issue is well documented, as mentioned in this ...
0 votes
1 answer
9k views

Kibana - "missing authentication credentials for REST request"

I have a elasticsearch cluster with xpack basic license, and native user authentication enabled (with ssl of course). I am attempting to set up kibana on a docker container but keep getting an error ...
  • 405
0 votes
0 answers
530 views

Elasticsearch systemd setup

I am configuring Elasticsearch 2.3.3 (yes, outdated) on CentOS 7.7. We were running it on CentOS 6 with InitV script. I followed this specification to create systemd script and get inspired with the ...
0 votes
1 answer
2k views

Can't access Kibana remotely - Can curl successfully on machine

I've installed Kibana on rhel and am I'm trying to access remotely on http://server-url:5601. I've installed elasticsearch and am able to access that remotely on http://server-url:9200 I've added ...
0 votes
0 answers
13 views

Adding context paths to ELK log analytics?

I'm looking to install the ELK stack on my application server by following the instructions here: I can see you can then access the various ELK services on localhost via certain ports. Is there a way ...
0 votes
1 answer
521 views

Grok filter is not working properly

I have Filebeat-7.1 installed in a Debian server, this Filebeat send data from files in this Debian server to server with Logstash 7.6 , here are the files config Filebeat.yml: #=====================...
  • 1
0 votes
2 answers
228 views

Kibana and Logstash: When is a field not a field?

I've enabled the grok filter in our logstash configuration in order to parse logs from HAProxy using the %{HAPROXYHTTP} and %{HAPROXYTCP} patterns. This seems to work great and viewing the details for ...
  • 41.3k
1 vote
0 answers
434 views

Scaling Elasticsearch down to single-node

Is it possible to scale Elasticsearch from multiple nodes down to one node? I have a 3-node cluster that is way overkill for the amount of data being logged. To scale it down, I set "cluster.routing....

1
2 3 4 5
8