How can I remove a label and then rename another one to replace it?

Question

I have two different metrics that I am trying to combine in a PromQL expression to write an alert. One of the metrics, say is_up represents if a node is up. The other metric is_active represents if something on that node is running.

Each of them have labels that can be used to compare them, but the labels are called different things on each. So on is_up we have label_a="node_id" and on is_active we have label_b="node_id" where the node_ids are the same.

If the label names were the same, e.g. label, I could write a query that looks like:

count(is_up) by (label) unless count(is_active) by (label)

But this isn't possible. I can't relabel it in a global config as there is already a label on is_up that is called label_b but it has a different format :/

So my current thinking is to (using PromQL, so at evaluation time) remove the label_b from is_up and rename the label label_a to label_b

I saw this answer previously about this topic: https://stackoverflow.com/a/60894489/16546473 but I'm not sure that it applies to me as it's just talking about renaming a label, but I need to

1. remove label_b from is_up, then
2. rename label_a in is_up to label_b so that it can be directly compared to label_b in is_active

Let me know if this makes sense

Answer 1

I managed it like this:

sum without (label_a) (label_replace(sum(is_up) without (label_b), "label_b", "$1", "label_a", "(.*)"))

Answer 2

The full solution is the following:

count(is_up) by (label_a)
  unless on (label_a)
label_replace(
  count(is_active) by (label_b),
  "label_a",
  "$1",
  "label_b",
  "(.*)"
)

It works in the following way:

1. The count(is_up) by (label_a) counts is_up series grouped by label_a. See docs for count() aggregate function.
2. The count(is_active) by (label_b) counts is_active series grouped by label_b.
3. The label_replace(...) copies the label_a value to label_b for all the series returned at step 2. Note that the time series now have two identical labels - label_a and label_b. See label_replace() docs.
4. The unless on (label_a) instructs returning series from step 1 only if there are no matching series from step 3 with the given label_a value. See docs for unless operator and docs for on() modifier.