3

I'm setting up Grafana v8.3.4 alerts. If I use a template like the following

{{ define "test_message" }}

{{ range .Alerts.Firing }}
{{ print .ValueString }} 
{{ end }}

{{ end }}

enter image description here

And test the e-mail notification using this template and the predefined message:

enter image description here

I get:

[ metric='foo' labels={instance=bar} value=10 ]

In the example above there's only one metric, but there could be more.

Question: Is there any way to loop through the returned metrics and display it as a table in the e-mail?

It would look something like:

Metric Name Value
foo 10
x 22
y 7

...

[UPDATE 6/24/2022] @Luis Almeida's answer led me to the right path, so I'm marking it as the correct answer even though my final solution was a little different.

Here are a couple of other links that also helped:

How to template annotations and labels - Grafana's Official Documentatio n Unified Alerting Grafana 8 | Prometheus | Victoria | Telegraf | Notifications | Alert Templating

EduardoCMB
  • 392
  • 2
  • 17
  • Hi @EduardoCMB can you post your final solution? – SergkeiM Feb 03 '23 at 13:02
  • Hi @Froxz... it's been a while, and I'm not sure if I switched gear in the end... let me see what I can find it. I'll be in touch soon. – EduardoCMB Feb 09 '23 at 16:43
  • @EduardoCMB, please advise how to delete the Value field from the alert notification email and to only keep the Labels field as displayed on the screenshot. https://i.stack.imgur.com/mNsl1.png (or https://ibb.co/rmdNwYz if another link doesn't work). – Eugene_S Apr 26 '23 at 22:08

1 Answers1

2

So what you basically have are multiple alerts (which one has a metric defined by [ metric='foo' labels={instance=bar} value=10 ]) and you want to pull all this alerts in a table?

I'm using a classic condition with a median function in order to remove eventual outliers btw. If you want to access the value in:

  • the first condition you should put {{ $values.B0.Value }}
  • the second condition you should put {{ $values.B1.Value }}
  • the n-th condition you should put {{ $values.Bn.Value }} classic condition If you want to access the a normal query value I think the variable {{ $values.A.Value }} where A is the query's name, but since grafana documentation for versions below 9 disappear I cannot confirm that

You can use annotations to give a more readable metric's name as depicted in the image above.

I believe that html tags are escaped since html tags appears on the template as text. So now that you have the annotations with the given values you have to edit the default email template that is in /usr/share/grafana/public/emails/ng_alert_notification.html

Warning: Do a backup of this file just in case something goes wrong

In the image bellow is how annotations are render in the email, now you just have to substitute that code with some code that renders a table. I believe that now it's fairly simple to do it yourself. (I didn't put the code here since I am not that comfortable with golang). If you want something different you can analise this file and change as you want. ng_alert_notification Note: To take effect the changes done in ng_alert_notification.html you have to restart grafana server

Good looking HTML and CSS example

Note: I'm using v8.5.2

Luis Almeida
  • 101
  • 7