Stack Overflow
Stack Overflow

Questions tagged [elk]

Questions about the ELK stack, which gives you the ability to aggregate logs from all your systems and applications, analyze these logs and create visualization.

ELK is a stack of the three open source projects - , and

The ELK stack gives you the ability to aggregate logs from all your systems and applications, analyze these logs and create visualization.

Useful links:

1095 questions
2
votes
2 answers

Crontab is not running

I tried to schedule crontab for automate kibana dashboard data export, but it's not working. crontab -e is * * * * * /home/rehan/Installation/SIEM/ELK%20/Other/test.sh test.sh file has execution permission but the script is not get run every…
rehan
  • 469
  • 1
  • 7
  • 17
2
votes
0 answers

How can I search custom header in Elasticsearch apm?

I have a quick question I can not use custom headers for searching by below code. But it is not working. http.request.headers["correlationId"]: "d7fc64cf-d176-4a17-9a8f-66c21b48265d" is it possible by writing any query for searching above code in…
loki
  • 2,926
  • 8
  • 62
  • 115
2
votes
1 answer

Pull logs from remote server into elasticsearch

The short question is: Is it possible to pull logs (within logfiles) from a remote server and ingest them into the ELK stack. The long story is the following: We have a setup with a DMZ which is publically facing We have an intranet environment…
pfeigl
  • 457
  • 5
  • 12
2
votes
1 answer

ELK - Removing old logs viewable in Kibana

I have managed to process log files using the ELK kit and I can now see my logs on Kibana. I have scoured the internet and can't seem to find a way to remove all the old logs, viewable in Kibana, from months ago. (Well an explaination that I…
alexmcc42
  • 27
  • 1
  • 2
  • 5
2
votes
1 answer

AWS CloudWatch Logs to ELK

I am planning to copy the AWS CloudWatch Logs to ELK and want to use Kibana Dashboard to visualise the logs. One option is to stream the logs from CloudWatch to ELK. https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_ES_Stream.html But I…
Dattatray
  • 1,745
  • 1
  • 21
  • 49
2
votes
0 answers

Kibana Error : The aggregations key is missing from the response, check your permissions for this request

I am not much familiar with ELK stack. We monitor a particular server health using ELK stack. We have Metricbeat which sends statics and based on that we have index created in Elastic. Kibana visualization is created based on this. But from last…
Akhil K
  • 21
  • 1
  • 3
2
votes
1 answer

Combine request and response based on field in log - ELK

We have ELK setup with filebeat, logstash and elasticsearch and kibana. I need aggregate request and response at the logstash. I have configured pipeline configuration as below. now log aggregation working without any issue if i use single worker…
techzone4all
  • 123
  • 2
  • 10
2
votes
3 answers

How to rotate ELK logs?

I have indexes around 250 GB all-together in 3 host i.e. 750 GB data in ELK cluster. So how can I rotate ELK logs to keep three months data in my ELK cluster and older logs should be pushed some other place.
Sourav
  • 3,025
  • 2
  • 13
  • 29
2
votes
2 answers

Failed to send join request to master in Elasticsearch, Unknown NamedWriteable [org.elasticsearch.cluster.metadata.MetaData$Custom][licenses]]

We have a long running single node ELK cluster running (master/data). I have decided to add additional data node. However Im getting the below error on the data node 30.X.XXX}{172.30.X.XXX:9300}{ml.enabled=true}], reason…
Karthik Varma Koppella
  • 21
  • 1
  • 7
2
votes
1 answer

logstash Issue sending data to elasticseach cluster

I have upgraded ELK stack into 7.4 version ( filebeat, logstash, elasticalert, kibana). I am using elasticsearch cloud. Once after the upgrade, following error shows in logstash log file. but few of records can see in…
techzone4all
  • 123
  • 2
  • 10
2
votes
1 answer

Match version number parts in grok

I want to split a version string (e.g. 2.3.5) into three fields(major.minor.patch) using the grok filter. What I tried: %{NUMBER:major}.%{NUMBER:minor}.%{NUMBER:patch} Result from Grok debugger: No Matches What I expected: { major: 2, …
Wenyi Li
  • 47
  • 7
2
votes
1 answer

Can I set "data" properties on a Jenkins Logstash message?

I'm sending build logs to Logstash via the logstashSend method at the end of a Jenkins declarative pipeline. The logs are being written to Logstash and I can query them in Kibana. The "data" section of the message contains what looks like a…
Richard Schaefer
  • 525
  • 3
  • 13
  • 45
2
votes
0 answers

logstash timestamp date filter

I would like to change the @timestamp to the time from one of my fields I have a field called Time which has the time I want to use for the @timestamp. This time though is in an inconvenient format "dd-MM-yyyy_HHmmss" The time is…
griffer98
  • 71
  • 1
  • 1
  • 5
2
votes
1 answer

What is the type of my document field in ingest pipeline?

I have a Docker -> Filebeat -> Elasticsearch pipeline for logs. I am using Elasticsearch ingest pipelines to process my logs(after Filebeat sends them). In my logs, there is a message field and payload field. Here is the fun part: Payload is…
Michal Fašánek
  • 513
  • 5
  • 17
2
votes
0 answers

Logstash configuration to modify exception stacktrace

My application send log events to logstash using log4j2 socket appender and JsonLayout. In kibana I want to show all other properties in json format other than stacktrace. log4j2.xml configuration
user3153309
  • 151
  • 1
  • 6
1 2 3
…
72 73