2

I would like to monitor all ELK service running in our kubernetes clusters to be sure, that is still running properly.

I am able to monitor Kibana portal via URL. ElasticSearch via Prometheus and his metrics (ES have some interested metrics to be sure, that ES is working well).

But exist something similar for Filebeat, Logstash, ... ? Have these daemons some exposed metrics for Prometheus, which is possible to watching and analizing it states?

Thank you very much for all hints.

Adavan
  • 63
  • 2
  • 6

2 Answers2

2

There is an exporter for ElasticSearch found here: https://github.com/prometheus-community/elasticsearch_exporter and an exporter for Kibana found here: https://github.com/pjhampton/kibana-prometheus-exporter These will enable your Prometheus to scrape the endpoints and collect metrics.

We are also working on a new profiler inside of OpenSearch which will provide much more detailed metrics and fix a lot of bugs. That will also natively provide an exporter for Prometheus to scrape : https://github.com/opensearch-project/OpenSearch/issues/539 you can follow along here, this is in active development if you are looking for an open-source alternative to ElasticSearch and Kibana.

Jonah Kowall
  • 577
  • 2
  • 7
-1

Yes, both the beats and logstash have metrics endpoint for monitoring.

These monitoring endpoints are built to be consumed using metricbeat, but since they return a json you can use other tools to monitor it.

For logstash the metrics endpoint is enabled by default, listening on localhost at port 9600, and from the documentation you have these two endpoints:

  • node
  • node_stats

For the beats family you need to enable it as if you would consume the metrics using metricbeat, this documentation explains how to do that.

Then you will have two endpoints:

  • stats
  • state

So you would just need to use those endpoints to collect the metrics.

leandrojmp
  • 7,082
  • 2
  • 19
  • 24
  • Thank you for your answer. It seems, that it is not accurate, but it gives me some idea, which looking for. I found [logstash_exporter](https://github.com/sequra/logstash_exporter), which I would like to try. Your answer pushed me closer to the solution. – Adavan Jun 25 '21 at 09:46