Questions tagged [elk]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
36 views

I used elk+redis+filebeat to build the logging platform, and now after redis is replaced by codis, logstash reports???

I used elk+redis+filebeat to build the logging platform, and now after redis is replaced by codis, logstash reports: # [2020-06-18T11:20:54,146][WARN ][logstash.inputs.redis] Redis connection problem {...
0
votes
1answer
70 views

Ship Logs from application server to ELK server with beats

I am building log analyzer for production environment. My scenario is, i need to setupelasticsearch,Logstash and kibana on one centos7 server which is going to use ELK server another one is apache ...
0
votes
1answer
65 views

Export Google Cloud SQL slow logs to ELK stack

I stumbled upon an issue and decided to ask for advice and eventually find someone with the same business need (and problem). Summary - we’ve recently migrated the SQL service of one of our clients ...
0
votes
0answers
25 views

Get a list of Kibana dashboards

I am trying to get a list of Kibana dashboards. Here is my command: curl -u 'elastic:xxxxx' -XGET --cacert /opt/ssl-certs/client-ca.cer -s https://x.x.x.x:9201/.kibana/_search?type=dashboard&...
0
votes
0answers
18 views

How to filter in kibana on input fields set in filebeat?

I've got filebeat setup sending logs to kibana. I have the apache module enabled which is sending the httpd error and access logs to kibana. I can filter on these by looking at the event.dataset field ...
0
votes
0answers
37 views

Monitoring tomcat logs (catalina.out) with Elastic stack and viewing in Kibana?

I've setup and installed Kibana and elastic search on my redhat vm. I then followed the fairly simple instructions to install filebeat with the apache module to gather the information on the apache ...
0
votes
1answer
28 views

Can't access Kibana remotely - Can curl successfully on machine

I've installed Kibana on rhel and am I'm trying to access remotely on http://server-url:5601. I've installed elasticsearch and am able to access that remotely on http://server-url:9200 I've added ...
0
votes
0answers
9 views

Adding context paths to ELK log analytics?

I'm looking to install the ELK stack on my application server by following the instructions here: I can see you can then access the various ELK services on localhost via certain ports. Is there a way ...
0
votes
0answers
37 views

What do the various “fit” algorithms mean in Kibana?

In Timelion visualisations in Kibana there is the option to interpolate/upsample an infrequent measurement, i.e. "invent" samples in the empty timestamps between known values to fill out a line plot. ...
0
votes
0answers
187 views

ELK - Logstash not picking up syslog events

I'm setting up a ELK cluster using Centos 8 and version 7.4 of Elasticsearch, Logstash and Kibana. My issue is with Logstash not picking up the events coming through syslog. Configuring Logstash to ...
0
votes
0answers
50 views

APM PHP Agent Setup

Im having a really hard time setting up my APM PHP Agent on a test EC2 server, can anyone please show me how to configure it properly to send log data to my ELK Server. I tried asking this question on ...
0
votes
0answers
80 views

Squid Proxy With Kibana

0 So i have kibana/ELK setup on my server and i have another server which has squid installed and running fine. BUT. for some reason when i try to https to kibana (which i can do fine without the ...
0
votes
0answers
39 views

ELK Cluster Minimum Node Count

Can we build ELK Cluster with one master and one data node, 2 nodes cluster in total? I need replication of shards. Just in case one of the node is down, does master also keep a copy of shards or is ...
0
votes
0answers
22 views

Trying to find source machine sending logs to ELK

So I have a machine receiving logs from multiple servers. There's a particular log that gets repeated a lot and I can't find out the source machine that is generating it. I'm fairly new to ELK, and, ...
2
votes
0answers
169 views

Logstash to aggregate logs into ElasticSearch

I am trying to aggregate linux logs using rsyslog into Logstash/ElasticSearch running in EKS. filebeat is already running in EKS to aggregate Kubernetes container logs. I have configured rsyslog ...
0
votes
1answer
236 views

logstash not able to upload data to elasticsearch even the pipeline started

I am using elasticsearch 7.1.1 and logstash 7.1.1. I am trying to upload a log file to elastic search using grok filter. Pipeline is getting started, but data is not getting uploaded. Here is my ...
0
votes
1answer
43 views

Using Elasticsearch or Logstash output?

I have been going through a few tutorials on using beats to send data to elasticsearch. I noticed that some tutorials prefer to use logstash as the output which then outputs to elasticsearch. Some ...
0
votes
1answer
108 views

Parsing or Reformatting Logs before feeding them to Splunk or Elastic Search

I have very complex log messages, that I want to reduce to the most important fields in order to save quota. The log messages are multiline and there is a lot of redundant information in them. A ...
1
vote
2answers
641 views

Line breaks in PHP's stack trace in nginx error logs disturbing logstash analysis

I am using nginx with PHP-FPM and ELK as log file analysis. When a PHP script causes an error the interpreter the error will be send back to nginx and nginx puts the error into the error.log file. ...
0
votes
1answer
940 views

rsyslog 8.24 and multiple omfwd

I want to forward *.* to remote host via TCP/IP. Also, I have local0 facility where messages are plain JSON messages and they has to be forwarded to same host, but other port (and uses same ...
2
votes
0answers
2k views

Parsing JSON event in Logstash

I have log in following format, it is a plain json with nested fields. { "level": "info", "message": { "req": { "headers": { "host": "localhost:8080", ...
0
votes
1answer
80 views

logstash charset error (gelf)

I had some trouble with Logstash and Nginx (all running on Docker). On my docker-compose file i was trying to enable logging from Nginx container to Logstash: nginx: container_name: nginx image: ...
1
vote
1answer
7k views

Filebeat can't connect to logstash on another server

Filebeat (11.11.11.11) can't connect to logstash (22.22.22.22) on another server (connection reset by peer). But filebeat services from other servers can do it. Also I can connect from this server(11....
0
votes
1answer
990 views

Retrieve pfSense/freeBSD logs with elk

I am attempting to centralize logs from different systems. I installed the Elastick Stack (Elasticsearch, Logstash, Kibana) and WAZUH OSSEC on one server (named elk). I have installed the OSSEC ...
0
votes
1answer
42 views

Import Elasticsearch 2.X data to 6.X?

I've got some data/logs from Elasticsearch 2.4 & I've got a new Elasticsearch 6.X running. What's the best/correct way to import data from 2.X to 6.x? Thanks!
2
votes
0answers
552 views

filebeat makes a lot of I/O

We have filebeat on few servers that is writeing to elasticsearch. We can see that it is doing a lot of writes: PID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 353 be/3 ...
2
votes
1answer
170 views

ELK logstash and core grok patterns

I'm evaluating the ELK stack with filebeat & logstash across a diverse range of applications/ servers. I understand the power of customising my own grok patterns for each application/log, but to ...
0
votes
1answer
775 views

Info sent from Logstash via elastic output not showing in Kibana, but file output works fine - what am I doing wrong?

I have an "ELK stack" configuration and, at first, was doing the standard 'filebeat' syslog feeding from logstash with the elasticsearch output plugin. It worked just fine. Now I have added a TCP ...
0
votes
1answer
6k views

Auto delete elasticsearch data older than 30 days

I have setup a ELK stack to collect logs at central server. It is working perfectly. But by default it is holding elasticsearch index/data permanently. We just want to maintain the data for 30Days. ...
-1
votes
1answer
411 views

How to display filebeat logs as a single file in kibana?

I have used ELK for displaying my middleware logs in kibana. for that I am using multiline patterns which mentioned below, after those logs displaying seperatly like separate expandable. How to ...
0
votes
1answer
378 views

Logstash creating many tcp connections for single hosts

Logstash: 5.3.0 nxlog: nxlog-ce-2.9.1716 Checking nestat I have like 300 established connections even though I have only about 50 hosts. I have like 40 connections for some hosts when I the client's ...
1
vote
1answer
4k views

cannot validate certificate - doesn't contain any IP SAN

I am currently in the process of installing ELK ( ElastricSearch, LogStash & Kibana) stack. My ELK server IP address is 172.29.225.32. Elastic Search config is :: # -----------------------------...
0
votes
2answers
1k views

Troubleshooting rsyslog integration with ELK stack

I'm trying to configure rsyslog to send logs to logstash on CentOS. So I was following the tutorial. However, after setting up, nothing happens. Everything started ok, not error occurred but no logs ...
0
votes
1answer
876 views

HAProxy in front of logstash broken pipe

I'm setting up ELK stacks with loadbalancing. Haproxy works fine for ES and Kibana but I'm having issues with Logstash. Here's the haproxy configuration: frontend logstash bind 0.0.0.0:5000 ...
0
votes
1answer
2k views

Graylog stream getting events, but is empty

I've started sending Palo Alto logs to Graylog, and a stream rule picks them out by matching "Palo Alto" in a "tags" field (which is how all my stream rules are; a front-end Logstash instance does the ...
4
votes
1answer
1k views

Externally visible Kubernetes Service on Azure

I am deploying an ELK stack on a Kubernetes(v1.4.5) cluster on Azure. This is the configuration that creates the Kibana Service and Deployment. # deployment.yml --- apiVersion: v1 kind: Namespace ...
1
vote
0answers
459 views

Logstash syslog filter not applying to logs?

I'm looking through some syslog logs files in my ELK stack and noticed that all the syslog_severity fields are 'notice', when I can verify in the log files that they are not 'notice'. Seems like ...
1
vote
0answers
232 views

Logstash filter: syslog_pri always defaulting to notice?

I'm looking through some syslog logs files in my ELK stack and noticed that all the syslog_severity fields are 'notice', when I can verify in the log files that they are not 'notice'. Seems like ...
3
votes
1answer
749 views

Monitoring and alerting tools for small businesses in 2016 [closed]

This is a kind-of-recurring question, yet the closest one I could find was asked 7 years ago, which was pretty much a different time. I run a small business and we host multiple small-to-medium ...
2
votes
1answer
2k views

Trouble bringing in CloudWatch data to Logstash

Just getting started with a new ELK setup (never used it before, just trying to learn it). I have Logstash 2.2.4 running on ubuntu 14.04 LTS. After putting a yaml file down with my monitor user's ...
1
vote
1answer
478 views

Using ELK X-pack for general purpose alerts and alarms

The X-pack package from Elastic is fully integrated with Elasticsearch and Kibana to provide (among other things) an alarm reporting platform. I believe that the regular use case is to build that ...
1
vote
0answers
31 views

How does Docker Daemon handle large log output?

I have a number of server applications running in Docker. The output is configured to go to an elk stack. I've had a number of troubles with the elk stack and am considering going back to vanilla ...
1
vote
1answer
2k views

elastic's snapshot and restore module repository_exception

I'm using elk-docker and trying to follow Snapshot And Restore | Elasticsearch Reference [2.4] | Elastic and getting following error: # curl --request PUT --data '{ "type": "fs", "settings": {"...
8
votes
1answer
8k views

How to forward Docker container logs to ELK?

I would like to know what is the easiest way to forward my docker container logs to an ELK server, so far the solutions I have tried after having searched the internet didn't work at all. Basically ...
3
votes
1answer
11k views

elk stack error “unable to fetch mapping do you have indices matching the pattern”

i am trying to setup ELK stack with collectd on Ubuntu 16.04 LTS (so pretty much latest version of stack available) kibana is behind nginx proxy (followed this guide https://www.digitalocean.com/...
0
votes
1answer
270 views

Can I setup logstash with windows generated certs instead of openssl?

Going through this tutorial. They use openssl to generate certs to use with logstash ie this command: sudo openssl req -subj '/CN=ELK_server_fqdn/' -x509 -days 3650 -batch -nodes -newkey rsa:2048 -...
0
votes
2answers
419 views

ELK stack on AWS and web app on Digital Ocean, how and can I grab logs from the later?

I have a python website hosted on digital ocean and ELK stack - Elastic Search, Kibana, Logstash - on AWS. I'm trying to understand how to feed the logs files from my python web app on digital ocean ...
0
votes
2answers
604 views

Is is possible to pass logs to ELK stack via a ZeroMQ connection?

I have just started learning about the ELK stack. I know that Logstash can take input from several different sources via filebeat client and put that in ElasticSearch. Is it possible to pass log data ...
0
votes
2answers
959 views

rsyslog TLS on AIX

I am trying to forward logs with rsyslog (8.4.2) on AIX 7.1 and I need to encrypt with TLS. However it seems I need rsyslog-gnutls package. I can't seem to find this package anywhere for AIX. Best I ...
2
votes
0answers
290 views

Is there any way to get log events from crashlytics / fabric into ELK or a SaaS metrics platform like datadog?

If you use a stack like ELK or datadog for collecting server-side logs and events, how do you integrate mobile-side metrics? Is there any way to get these out of crashlytics directly, or does this log ...