How to ignore empty dataseries in prometheus

Question

Calculating the maximum quantile over all dataseries is a problem for me:

query

http_response_time{job=~"^(x|y)$", quantile="0.95",...}

result

http_response_time{job="x",...} 0.26
http_response_time{job="y",...} NaN

This is how I would try to calculate the maximum:

avg(http_response_time{job=~"^(x|y)$",...})

Now the result will be "NaN". How can I ignore the "NaN" result (from the result section)?

UPDATE 0

The metric is a self made summary-metric.

UPDATE 1

Using prometheus version 1.8.

Can you share more information about the metric in question, and where it's coming from? — brian-brazil, Dec 19 '17 at 12:54
http_response_time{job="x",...} 0.26 http_response_time{job="y",...} NaN returned by http your node_exporter? — Nikolay Baranenko, Dec 19 '17 at 12:56

svenwltr · Accepted Answer · 2017-12-19T13:31:04.103

23

I didn't try this one with NaN, but you can simply filter by values with binary operators. Since NaN mathematically doesn't equal NaN you could try this trick (since a response time should be always positive):

avg(http_response_time{job=~"^(x|y)$",...} >= 0)

edited Dec 19 '17 at 13:31

answered Dec 19 '17 at 13:11

svenwltr

17,002
12
56
68

1

Good try. But "http_response_time{ quantile="0.95"} != NaN" does not filter/remove the "NaN" results. – eventhorizon Dec 19 '17 at 13:16
Looks good. Will check it in detail before setting your answer as "correct". THX! – eventhorizon Dec 19 '17 at 13:53
By definition, NaN boolean operations are strange. For example "NaN != NaN" returns true in Javascript and pretty much every other language. – Bryan Larsen Dec 13 '19 at 17:19

How to ignore empty dataseries in prometheus

1 Answers1