1

I am evaluating VictoriaMetrics for an IoT application where we sometimes have gaps in a series due to hardware or communication issues. In some time series reporting situations it is helpful for us to interpolate values for the missing time intervals. I see that MetricsQL (which extends PromQL) has a keep_last_value() function that will fill gaps by holding the last observed value until a new one appears (which will be helpful to us) but in some situations a linear interpolation between the values before and after the gap is a more realistic estimate for the missing portion. Is there a function in PromQL or MetricsQL that will do linear interpolation of missing data in a series, or is it possible to construct a more complex query that will achieve this?

Clarifying the desired interpolation

What I would like is a simple interpolation between the points immediately before and after the gap; this is, I believe, what TimescaleDB's interpolate() function does. In other words, if my time series is:

(1:00, 2)
(2:00, 4)
(3:00, NaN)
(4:00, 5)
(5:00, 1)

I would like the interpolated 3:00 value to be 4.5, half way between the points immediately before and after. I don't want it to be 6 (which is what I would get by extrapolating from the points before the missing one, ignoring the points after) and I don't want whatever value I would get if I did linear regression on the whole series and interpolated at 3:00 (presumably 3, or something close to it).

Of course, this is a simple illustration and it's also possible that the gap could last more than one time step. But in that case I would still like the interpolation to be based solely off of the points immediately before and immediately after the gap, ignoring the rest of the series.

valyala
  • 11,669
  • 1
  • 59
  • 62
mactyr
  • 463
  • 3
  • 16

1 Answers1

3

Final answer

Use the interpolate function, now available in VictoriaMetrics starting from v1.38.0.

Original suggestion

This does not achieve the exact interpolation requested in the revised question, but may be useful for others with slightly different requirements

Try combining predict_linear function with default operator from MetricsQL in the following way:

metric default predict_linear(metric[1h], 0)

Try modifying the value in square brackets in order to get the desired level of interpolation.

mactyr
  • 463
  • 3
  • 16
valyala
  • 11,669
  • 1
  • 59
  • 62
  • 1
    Interesting idea, but reading up on `predict_linear` it doesn't appear that it would do the kind of interpolation I'm looking for. I've added a section to the question clarifying exact behavior that I'm seeking (which I believe is the same as TimescaleDB's `interpolate()`, for reference). Would `predict_linear` achieve this? Or is there another way? – mactyr Jul 01 '20 at 18:00
  • 2
    Thanks for the updated question! I just added `interpolate` function into VictoriaMetrics - see [this commit](https://github.com/VictoriaMetrics/VictoriaMetrics/commit/5d3db3ff7cd137ab81bfedcbb0e12ccd1a670479) for details. It will be included in the next release. In the mean time you can build VictoriaMetrics from sources according to [these docs](https://victoriametrics.github.io/#how-to-build-from-sources) – valyala Jul 02 '20 at 11:58
  • 1
    Excellent. I look forward to trying it out. – mactyr Jul 02 '20 at 22:59
  • 2
    FYI, the `interpolate` function is available in VictoriaMetrics starting from [v1.38.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases) – valyala Jul 08 '20 at 18:46