15

I have a Grafana dashboard panel configured to render the results of a Prometheus query. There are a large number of series returned by the query, with the legend displayed to the right. If the user is looking for a specific series, they have to potentially scroll through all of them, and it's easy to miss the one they're looking for. So I'd like to sort the legend by series name, but I can't find any way to do that.

My series name is a concatenation of two labels, so if I could sort the instant vector returned from the PromQL query by label value, I think Grafana would use that order in the legend. But I don't see any way to do that in Prometheus. There is a sort() function, but it sorts by sample value. And I don't see any way to sort the legend in Grafana.

MidnightJava
  • 1,927
  • 2
  • 18
  • 38

2 Answers2

15

As far as I know, You can only use the function sort() to sort metrics by value.

  • According to this PR, Prometheus does not intend to provide the function sort_by_label().
  • According to this Issue, Grafana displays the query results from Prometheus without sorting.
  • According to this Issue, Grafana supports sorting by value when displaying legend.

In Grafana 7, Prometheus metrics can be transformed from time series format to table format using the Transform module, so that you can sort the metrics by any label or value.

LeoHsiao
  • 196
  • 2
  • 6
  • So after I have used a Transform to get a table, how do I choose which columns of the table I want to sort on? – Patrick Szalapski Apr 12 '21 at 20:51
  • Just like a regular table, you can click on the column name of the table and it will be sorted in ascending or descending order. In addition, Transform supports changing column names, filtering, and calculating statistics. – LeoHsiao Apr 14 '21 at 03:26
  • 2
    The position of the prom maintainers seems to have softened a bit and you might get lucky in the future with the following PR https://github.com/prometheus/prometheus/pull/11299 – Rick Rackow Apr 28 '23 at 13:16
4

Prometheus doesn't support sorting results by series names as mentioned in https://stackoverflow.com/a/65353296/274937 . If you still need sorting results by metric name or by arbitrary set of labels, then take a look at VictoriaMetrics. It provides sort_by_label and sort_by_label_desc functions. For example, the following query sorts time series returned by query by metric names:

sort_by_label(query, "__name__")

The __name__ refers to metric name. See these docs for more details.

valyala
  • 11,669
  • 1
  • 59
  • 62
  • 2
    VictoriaMetrics is 100% retro-compatible with Prometheus but (IMHO) with superior quality-of-life improvements such as this. =) – lapo May 17 '22 at 16:35