0

I'm making a Grafana dashboard to display the performance of the canary application. My problem is I need to find out which instance is the canary one (blue or green).

Canary stack will always create one instance either blue or green so I can see the count of the instance using the below query but can't make to display the value of the bound metric.

(count(bound(cfstack=".Blue.")) == 1) or ( count(bound(cfstack=".Green.")) == 1)

How can I express the following in PromQL?

if ( count(bound(cfstack=".*Blue.*")) == 1 )
   cfstack_val=".*Blue.*"
else 
  if ( count(bound(cfstack=".*Green.*")) == 1 )
    cfstack_val=".*Green.*"

bound(cfstack="${cfstack_val}")
Roopchand
  • 33
  • 5

1 Answers1

0

Try the following query:

(bound{cfstack=~".*Blue.*"} and on() (count(bound{cfstack=~".*Blue.*"}) == 1))
  or
(bound{cfstack=~".*Green.*"} and on() (count(bound{cfstack=~".*Green.*"}) == 1))

This query works in the following way:

  1. It selects a time series matching the bound{cfstack=~".*Blue.*"} only if there is only a single such a time series.
  2. It selects a time series matching the bound{cfstack=~".*Green.*"} only if there is only a single such a time series.
  3. It returns both results from steps 1 and 2 with the help of or operator.

See these docs for or and and operators.

See these docs for on() modifier.

valyala
  • 11,669
  • 1
  • 59
  • 62
  • Thanks!! It works. But I am facing another issue now... we have multiple regions in our environment so I have to check in each region and display the result. I tried the below query and it works fine when I select the specific location but when I select ALL it simply providing all instances (both canary and non-canary) (bound{cfstack=~".*Blue.*",location=${location} and on() (count by (location) (bound{cfstack=~".*Blue.*",location=${location}}) == 1)) or (bound{cfstack=~".*Green.*",location=${location}} and on() (count by (location) (bound{cfstack=~".*Green.*",location=${location}}) == 1)) – Roopchand Jul 15 '22 at 10:19
  • Just add `location` label name into every `on()` modifier: `on(location)`. In this case Prometheus will match time series by `location` label values when performing `and` operation. – valyala Jul 15 '22 at 11:00
  • Thanks for saving my time!! Sorry to bother u again.. Adding one more panel to show CPU usage of canary ((sum(rate(cpu_usage_total{cfstack=~".*Blue.*",location=~"${loc}"}[1m])) by (ip)) and on(loc) (count by (loc) (sum(rate(cpu_usage_total{cfstack=~".*Blue.*",location=~"${loc}"}[1m])) by (ip)) == 1)) or ((sum(rate(cpu_usage_total{cfstack=~".*Green.*",location=~"${loc}"}[1m])) by (ip)) and on(loc) (count by (loc) (sum(rate(cpu_usage_total{cfstack=~".*Green.*",location=~"${loc}"}[1m])) by (ip)) == 1)) Same problem, it works fine with one location but when selecting as ALL it says no data – Roopchand Jul 15 '22 at 13:14
  • Substitute `on(loc)` with `on(location)` everywhere in the query. Please read https://prometheus.io/docs/prometheus/latest/querying/operators/#vector-matching – valyala Jul 15 '22 at 15:23