Questions tagged [elasticsearch]

Elasticsearch is a flexible and powerful open source, distributed, real-time search and analytics engine.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
18 views

installed docker/kibana on Mac installed elasticSearcy remotely

am trying to connect local kibana on Mac to remote elasticSearch any FAQ/Instructions/Video? Sincere thanks. Billy
bi422's user avatar
  • 1
0 votes
0 answers
38 views

Connecting multiple PHP containers to a single shared Elasticsearch container

14 PHP containers php_1, ..., php_14 and 14 NGINX containers nginx_1, ..., nginx_14 are running on a server. The PHP container php_i is only connected to nginx_i and all NGINX containers are connected ...
M a m a D's user avatar
  • 101
0 votes
1 answer
34 views

How to set kubernetes secrest outside a container section

I have a generated kubernetes secret like this as it is shown e.g. in argocd: apiVersion: v1 data: ES_GUEST_PASSWORD: ++++++++ ES_GUEST_USERNAME: ++++++++ kind: Secret I now want to use it as an ...
Fabian's user avatar
  • 101
0 votes
1 answer
28 views

How to gather application logs which are generating in custom path using EFK

I created the elasticsearch and kibina setup which is running outside the kubernetes cluster and i am using fluentd to gather kubernetes logs and to send data to elasticsearch. I am running fluentd as ...
karthik's user avatar
  • 101
0 votes
1 answer
402 views

Fluent-bit in Kubernetes cannot input timestamp data properly

At my company, I built a K8s cluster with Terraform and configured a logging system with EFK (Elasticsearch, Fluent-bit, Kibana). k8s and Elasticsearch use AWS's EKS and Opensearch Servcie (ES 7.10), ...
HJ S's user avatar
  • 1
0 votes
1 answer
444 views

Could not contact DNS servers (fluent bit)

I am configuring the fluent bit connection with elasticsearch, I have 3 elasticsearch nodes and I want to have high availability, according to the following documentation it is done like this text, ...
Augusto Marega's user avatar
0 votes
0 answers
71 views

Fluentd logs not sent to Elasticsearch on K8s

I have a cluster in VirtualBox to learn kubernetes. I have a deployment that contains MySQL and phpMyAdmin. I created a DemonSet that has the fluentd image and collects the logs to transmit them to ...
VBE's user avatar
  • 1
0 votes
0 answers
246 views

Fluentd logs not sent to Elasticsearch

I have a cluster in VirtualBox to learn kubernetes. I have a deployment that contains MySQL and phpMyAdmin. I created a DemonSet that has the fluentd image and collects the logs to transmit them to ...
VBE's user avatar
  • 1
0 votes
0 answers
56 views

How do I shutdown a specific node in ElasticSearch?

I'm running ElasticSearch 7.3.2 and I found this page documenting how to shut down a node: https://www.elastic.co/guide/en/elasticsearch/reference/7.17/put-shutdown.html I tried running this command (...
Pablo Fernandez's user avatar
0 votes
0 answers
299 views

High CPU usage of kube-controller and kube-apiserver when increasing nodes of ECK node sets

I am trying to run an Elastic Cluster on Kubernetes (Minikube). It basically works, but only with one server per node. As soon as I try to set up > 1 node per node set, the cpu usage of kube-...
xm22's user avatar
  • 1
0 votes
0 answers
503 views

Running Opensearch behind a proxy

when I configure nginx to redirect /monitor to opensearch, opensearch receives the requests and redirects the user to /app/something which is not under /monitor and thus fails (error 404). My nginx ...
mojovski's user avatar
  • 113
0 votes
0 answers
386 views

Could not communicate to Elasticsearch, resetting connection and trying again. EOFError (EOFError)[Fluentd using helm on kubernetes]

I'm installing elasticsearch, kibana and fluentd in kubernetes with helm chart Elasticsearch and kibana pods stand up smoothly, but fluentd pods don't stand up I get the following errors: 2023-06-13 ...
kernel's user avatar
  • 101
0 votes
0 answers
260 views

How to have multiple instances of filebeat load balance Netflow input?

I have a very high volume Netflow input stream, and I was hoping that I could run multiple instances of Filebeat and load-balance the Netflow traffic over the Filebeat instances, and then write to a ...
Rayne's user avatar
  • 211
0 votes
0 answers
269 views

Filebeat docker multiline

I'm using filebeat to retrieve log files to Elastic cloud. I'd like to put errors and java exception in one document instead of multiple for each line. I managed to make it work on text log files but ...
Ror's user avatar
  • 323
0 votes
0 answers
92 views

Logstash cloudwatch plugin behavior

We have an ELK stack set up on an ec2 instance that stopped working a month ago and I just got it working again (the problem was credentials with the cloudwatch logstash plugin). What is strange is ...
maximosis's user avatar
0 votes
0 answers
157 views

AWS centralized logging with single index for all log groups

I am setting up logging on AWS, based on official AWS Centralized logging (source). Cloudwatch log groups are added with command like: aws logs put-subscription-filter --destination-arn DEST --log-...
wtdmn's user avatar
  • 63
0 votes
1 answer
544 views

Can I use keycloak as an Idp for kibana installed using the ElasticSearch Operator in kubernetes?

I have setup a kubernetes cluster in a private network and I'm using a gateway machine for accessing the cluster. In the kubernetes cluster I have installed the elasticsearch operator and through that ...
user2835131's user avatar
0 votes
1 answer
8k views

Elasticsearch: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors (ca.crt matches on all nodes)

Note that this entire cluster was working fine up 'til I updated certificates. I'm on Elasticsearch 7.5. I updated my certificates using the certificate-util - created a server.yml with all my servers ...
Epsilon's user avatar
  • 11
2 votes
0 answers
621 views

rsyslog is not forwarding logs to elasticsearch

I'm trying to configure rsyslog to send logs to logstash and then forward them to elasticsearch. I have create a config file /etc/rsyslog.d/60-output.conf with the following content: *.* @localhost:...
Croviajo's user avatar
  • 141
0 votes
1 answer
634 views

SonarQube is not starting on SynologyNAS due to CONFIG_SECCOMP

SonarQube is unable to start on Synology NAS Docker due to the CONFIG_SECCOMP not being present on Synology NAS. Is there an option to ignore SECCOMP for SonarQube while it's starting ElasticSearch? ...
VeeeneX's user avatar
  • 103
0 votes
1 answer
294 views

Restricting Kibana to TLSv1.2

What is the setting that needs to be set in the kibana.yml file (or other file) to restrict Kibana to TLSv1.2 over HTTP? I believe that it uses TLSv1.0, TLSv1.1, and TLSv1.2 by default. I need to ...
Jay's user avatar
  • 1
1 vote
0 answers
69 views

Suricata / Filebeat / ELK - iptables tee - Create virtual hosts

I have an IDS setup as follow: Hardware / interfaces WAN <----(brwan)> ROUTER / AP <(br0)----> LAN \ -----(eth1)>...
Gabriel ROUSSEAU's user avatar
1 vote
1 answer
97 views

Handle 150 req/s and 2TB of logs for 7 days of retention available by API

I would like to get advise from experienced people to build a HA infrastructure to log 2To of data in JSON format every week. I need to have a retention time of 7 days and need to be able to requests ...
michbr010's user avatar
0 votes
1 answer
3k views

curl -X GET http://localhost:9200 return Access Denied

I am trying to install and setup Elasticsearch 8 on Centos 7 server, the service started but when I do curl -x GET HTTP://localhost:9200, or curl -x GET HTTP://127.0.0.1:9200 or curl -x GET HTTP://(...
Mai's user avatar
  • 1
0 votes
0 answers
1k views

Free storage space in amazon open search data node is low and others nodes has more than enough free space

In my AWS open search i am using 8 data nodes (r5.2xlarge.search) and 3 master nodes (r5.large.search) and each data nodes has allocated to a space of 200gb each. And also added my shard value as 4....
aks's user avatar
  • 57
0 votes
1 answer
462 views

Graylog - data from newest indice not returned on search

I have a Graylog server (newest version) collecting data from nginx. It´s been running for a while. I retrieve the collected data by querying my Elasticsearch cluster (v7 newest version) which ...
OpenHaus's user avatar
1 vote
0 answers
146 views

Amazon OpenSearch Service stably or less downtime?

Can anyone point out, What is the best way for the elastic search to run stably? We frequently get messages about it requiring updates and that an update requires some downtime. Could we have 2 ...
Santosh Baruah's user avatar
1 vote
0 answers
357 views

On Debian system, keepalived is consuming full cpu usage on both kibana nodes

I have master/slave kibana machines on Debian 10. From few days, cpu usage of keepalived daemon on both nodes are reaching to 100%. After keepalived restart it goes to normal, and this loops forever. ...
Roushan Jha's user avatar
0 votes
0 answers
37 views

How to fetch Individual output of different match_phrase inside one request

I am able to run below query which is giving response as total of two match_phrase. Inserting dummy data as below. POST /mod1/_bulk { "index" : { } } { "msg": "BA1" } { &...
abc's user avatar
  • 11
0 votes
1 answer
695 views

How to pass values to query externally

I have below similar logs. I have created dummy index and created mapping like below in dev-tools PUT new { "mappings": { "properties": { "@timestamp": { ...
abc's user avatar
  • 11
0 votes
1 answer
130 views

Can logstash "pull" data?

I have two servers. Server A is running Elasticsearch and Logstash. Server B is running filebeat and is also the server which contains all the logs I'm trying to analyse. Server A is behind a firewall,...
Aditya K's user avatar
  • 923
0 votes
0 answers
570 views

Finding source of high IOWait

I have a 5 node elasticsearch cluster. One host has had consistently high IOWait% (40+) for several weeks. The others seem fine (<10%). Notable: Host in question is not the cluster master New ...
ethrbunny's user avatar
  • 2,369
1 vote
1 answer
392 views

graylog 4.0 - Injector could not be created

Trying to set up graylog with official graylog2.graylog ansible role. Installed Elasticsearch (v7.10.2) cluster on three separate servers with official elasticsearch ansible role (node_1 - master, ...
Drew's user avatar
  • 123
1 vote
0 answers
1k views

My elastic search is not showing any data using elastic-agent

Surprisingly my elastic version 7.15 with basic license have installed the fleet server and then configured windows-policy with elastic-endpoint and windows integration. Have enabled all the rules; ...
Blason R's user avatar
1 vote
1 answer
602 views

Running Enterprise Search as a service

Elastic's Enterprise Search Installation Guide goes as far as starting the process locally - which is obviously not very stable. When installed from a .deb or .rpm, a service is actually set up and ...
RekaB's user avatar
  • 131
1 vote
1 answer
609 views

Is there a way to use the ingest attachment plugin with Elastic App Search

I'm working on a portal that hosts multiple types of documentation (HTML, PDF, PPTx, DocX) and makes them all searchable in one place. We could achieve this using the "standard" out of the ...
RekaB's user avatar
  • 131
1 vote
1 answer
826 views

Kibana does not trust Let's Encrypt CA in Kubernetes Helm Setup

I have a k8s cluster configured with calico as networking system. I'm running slightly customized versions of helm elastic/elasticsearch and elastic/kibana with security enabled. For security, I use ...
I. Shm's user avatar
  • 31
2 votes
1 answer
2k views

When to use a new index in Graylog (Elasticsearch)?

I have been searching for days now to find a good explanation on how indices are used by Graylog/Elasticsearch and when to create a new one. There is a lot of info about sharding indices but not much ...
Holly's user avatar
  • 133
0 votes
1 answer
261 views

Filebeat on ECK with AWS Module Fails Due To Metadata Error

We are running an Elastic Stack with ECK in EKS (7.8). We noticed that our filebeat daemonset and the AWS module were not processing logs from S3 and our SQS queues backing up. Looking at the logs on ...
JGG Xm8's user avatar
0 votes
1 answer
8k views

Elasticsearch - Can I delete a data-stream backing index?

I have a few data-streams configured in my Elasticsearch cluster (with ILM policy). Suppose I run into storage problems and etc. and I want to prematurely delete a few backing indices of the data-...
GKman's user avatar
  • 495
0 votes
1 answer
10k views

Kibana cannot be accessed through port 5601 in browser [duplicate]

I cannot get to kibana through web UI. I'm set the necessary ports in firewall-cmd, I've configured kibana.yml and elasticsearch.yml. I'm not seeing any relevant errors, or at least don't know if they ...
jFasaJr's user avatar
  • 121
0 votes
0 answers
69 views

Multiple Web Application - Cache Layer Design

I have multiple client e-commerce web applications running on a single VM, each web application is a node.js express application. The express web application communicates with the back end via API's ...
Tam2's user avatar
  • 111
0 votes
1 answer
520 views

How to stop ElasticSearch background task

According to the CLI help of ElasticSearch, you can start ElasticSearch in the background with elasticsearch -d or elasticsearch --daemonize. This is what I use all the time locally, as it seems to ...
bolino's user avatar
  • 275
0 votes
0 answers
35 views

Ubuntu server 'partially' responsive

I have an Ubuntu 18.04 server that has periodically unresponsive - I can ping it, and it appears that some https services are running (albeit v. slowly), however it hangs if I try to log in (on the ...
16shells's user avatar
0 votes
1 answer
66 views

realtime monitoring of linux server network

We have a few Linux server which act as Edge network, We want realtime (or near-realtime) information on network bandwidth, top IP addresses and network bandwidth per them and other network parameters ...
Yashar's user avatar
  • 151
0 votes
1 answer
128 views

How to display the logs of a VM on ELK stack which is running on another VM?

-ELK is running on localhost & successfully getting logs. -Both VM's are on same server. -OS on both VM'S is Ubuntu 18-04 LTS.
Baqir kazmi's user avatar
0 votes
1 answer
562 views

NGINX - Access port via SSL

I have ElasticSearch and NGINX set up on my server. Accessing it via SSL is failing. I can access ElasticSearch fine via: http://ipaddress:9200 http://mydomain.co.uk:9200 The domain was then set up ...
wharfdale's user avatar
  • 151
0 votes
0 answers
37 views

Storage balancing in elasticsearch cluster

I've a Elasticsearch cluster with 5 nodes. I've this used storage repartition: Node1: 76% Node2: 94% Node3: 88% Node4: 73% Node5: 74% How to do to storage balancing/leveling the used storage ? For ...
brimix's user avatar
  • 1
1 vote
0 answers
338 views

Correct reverse proxy to aws-es-proxy instance?

I installed a VPC with an Elasticsearch cluster with kibana on AWS. As that is VPC, it is not accessible publicly. Thus, to be able to access the kibana interface, I installed the https://github.com/...
transient_loop's user avatar
0 votes
1 answer
179 views

How to use filebeat to find password in log

I have an Elasticsearch 7.1 and i have configure filebeat to collect all log. I want to check if i have password in log. So anyone have an idea how can i find all password in log using filebeat. Thank ...
khaled's user avatar
  • 21

1
2 3 4 5
8