0

I am trying to get this metric from prometheus:

sum by(pod) (increase(application_input_total{service="$service"}[1m]))

It is somewhat working, but it has an margin of error in it. This metric shows the amount of requests that our Java application receives. When I do 1 call with Postman, Grafana shows it as 1,09 calls. When I do 20 calls, Grafana shows 22, 60 calls shows 67 in GF and so on. The higher the calls the bigger the difference get.

Someone knows how to solve this? I think it has something to do with the [1m] or with the Prometheus configuration, but not sure what it could be.

Beerus239
  • 45
  • 5
  • No, this is expected behavior of increase. [The increase is extrapolated to cover the full time range as specified in the range vector selector, so that it is possible to get a non-integer result even if a counter increases only by integer increments.](https://prometheus.io/docs/prometheus/latest/querying/functions/#increase) – markalex Jul 06 '23 at 14:36
  • Afaik, there is no universally good answer how this can be avoided. One of many attempts to explain this and and provide some workarounds you can find [here](https://stackoverflow.com/a/75828166/21363224). More generally - google it, its popular problem. – markalex Jul 06 '23 at 14:41

1 Answers1

0

Prometheus may return non-integer result from increase() over integer counter because of extrapolation. Additionally, Prometheus may miss a part of counter increase between the last raw sample just before the specified interval in square brackets and the first raw sample inside the interval. See this issue and this design doc for details.

If you want obtaining the expected integer results from increase() function without extrapolation and missing calculations, then try MetricsQL. It works like PromQL, but its' increase() function returns the expected results without issues mentioned above.

valyala
  • 11,669
  • 1
  • 59
  • 62