In my Prometheus alert configuration, I have some great summary annotations, containing different labels.
- alert: PodsPending
expr: kube_pod_status_phase{phase="Pending"} > 0
labels:
severity: high
annotations:
summary: "Namespace {{$labels.namespace}} has pods stuck in Pending"
- alert: Failures
expr: sum(application_responses_percent{status="500"}) by (environment, app, otherlabel) > 80
for: 5m
labels:
severity: high
annotations:
summary: "({{$labels.environment}}) Too many errors in {{$labels.app}}"
In Alertmanager, I would like to group the alerts by these summaries. This way the set of labels to group by can be dynamic per alert since they are part of the summary.
However, since the summary is an annotation, it cannot be directly used in Alertmanagers groupBy configuration as it can only use labels. How can I use summary, or another dynamic way, for grouping alerts?
I've tried to check metric-relabel-config if the annotation could be transformed to a label, but there is no annotation data there.
