7

Graphite, Elastisearch, Cloudwatch, Prometheus, InfluxDB are all supported backends for Grafana. I am creating an application with grafana front-end, but an not being able understand how these backends differ and which would be the best to use for my application (would prefer open-source). My use case is a static log file being imported from an external server which I want to parse and fill-in the DB to be consumed by grafana. The data can have up to 5000 time-series data points for about a 100 measurement. The database need not be distributed. I would be glad to get some tips on how I can select a backing database out of these. Thanks in advance!!

Aditya Kotwal
  • 101
  • 2
  • 5

2 Answers2

6

Good answer by Brian, but adding more. You have to think about monitoring as 3 sets of data, which unfortunately in OSS you need a large mix of tools and projects. The fundamentals of monitoring consist of metrics (numbers such as what Grafana is good at visualizing), events (unstructured text such as what ELK is good at collecting and visualizing), and metadata (relationships, configuration, and other elements which span the other two categories).

Most people will use different technology stacks for each.

Metrics:

  • Graphite - Old, but well proven (uses RRD data stores)
  • InfluxDB - Newest, but less proven. Probably the best technology today
  • Prometheus - Uses a proprietary binary file based data store.

Events:

  • ElasticSearch - Java based unstructured data store, needs a lot of hardware to scale.

Once you have the metrics and events to visualize you'll need a bunch of tools. On ElasicSearch the ELK stack is most common E = ElasticSearch L = Logstash (ingesting logs) K = Kibana (visualization). Another alternative is Greylog which is better than Kibana IMHO.

Grafana is common, but not the best visualization. Unfortunately, the OSS tools out there just aren't great with metrics today.

chrki
  • 6,143
  • 6
  • 35
  • 55
Jonah Kowall
  • 577
  • 2
  • 7
1

That sounds like an event logging use case, so Elasticsearch is probably your best bet.

For metrics uses cases Prometheus would be a good choice.

brian-brazil
  • 31,678
  • 6
  • 93
  • 86