3

I am monitoring 30 servers with Grafana using Prometheus. But I use template for this and I cannot use alert in the template. What would you recommend for this? The Monitor system does not work for me without the alert system. I would like to use one dashboard for all servers.

How do i convert variable ? Is there any example ? rate(node_network_receive_bytes_total{instance="$node",job="$job"}[$__rate_interval])*8

rohatgisanat
  • 708
  • 6
  • 15
MrTux01
  • 343
  • 3
  • 9

2 Answers2

2

I guess your question is how to create an alert rule for a query with variables. If so, then it is not possible at the moment. There is an open issue about it on GitHub, so the feature may appear at some point in future.

For now you can either create a dashboard/panel that do not use any variables, or you can setup Alertmanager and define your alert rules with prometheus.

anemyte
  • 17,618
  • 1
  • 24
  • 45
1

As a rough workaround you can remove the templating altogether and put the alerts either together(single query) or seperately for each instance (multiple query)

rate(node_network_receive_bytes_total{instance=~".*",job="<hardcoded_job_name>"}[$__rate_interval])*8

You would still see the the problematic instance in the mail/screenshot sent by Grafana.

This is a major drawback of Grafana and caused us to pickup other tools to fix the issue.

As mentioned by @anemyte, alerts with templates are currently not possible.

rohatgisanat
  • 708
  • 6
  • 15