1

During the last UK BST/GMT changeover I observed an odd thing happen on a production box where the 1 hour graphs generated by rrdtool where showing -nan and querying the rrd directly also returned the same which then lead to a load of alerts being generated for things appearing to be down.

The rrd was valid and was updating when checked tith rrdtool info;

rrd_version = "0003"
step = 60
last_update = 1572136453
header_size = 2912
ds[ds0].index = 0
ds[ds0].type = "COUNTER"
ds[ds0].minimal_heartbeat = 600
ds[ds0].min = 0.0000000000e+00
ds[ds0].max = 1.0000000000e+05
ds[ds0].last_ds = "554"
ds[ds0].value = 5.5084745763e+00
ds[ds0].unknown_sec = 0
ds[ds1].index = 1
ds[ds1].type = "COUNTER"
ds[ds1].minimal_heartbeat = 600
ds[ds1].min = 0.0000000000e+00
ds[ds1].max = 1.0000000000e+05
ds[ds1].last_ds = "0"
ds[ds1].value = 0.0000000000e+00
ds[ds1].unknown_sec = 0

However calling:

rrdtool graph dummy --start=end-300s DEF:x=timedelta.rrd:ds0:AVERAGE VDEF:xa=x,AVERAGE PRINT:xa:%lf

Returned;

0x0
-nan

If I extended the window of --start=end-300s to something like 300000s I got a valid response. 5.596822

Graphs rendered via PHP for the hour showed as blank and all the time divisions along the bottom vanished. However, I have 6 Hour and longer graphs generated from the same rrd with the same php code that where still updating. (e.g. the 6 hour graph was showing valid data but lagging behind by about 10min presumable due to the way its generated.

Have I found a bug, or is this some odd behaviour to be expected during the DST changeover? (as a note I don't remember seeing this when going from GMT -> BST, and I don't remember what happened last year).

rrdtool-1.4.8-9.el7.x86_64
rrdtool-devel-1.4.8-9.el7.x86_64
php-7.3.11-1.el7.remi.x86_64
php-pecl-rrd-2.0.1-6.el7.remi.7.3.x86_64

And yes, the system locale is correctly configured.

user3788685
  • 2,943
  • 5
  • 26
  • 45
  • Sounds to me like it might be a bug calculating the last 1-hour window at the point of changeover that results in a zero-size interval? If so then you might find that in the spring you get a 2-hour interval briefly... I've never tested for this but it seems plausible – Steve Shipway Oct 28 '19 at 05:59
  • very odd since all time calculations should happen in GMT in rrdtool ... for starters you might want to be using a current version of rrdtool though ... – Tobi Oetiker Oct 28 '19 at 07:36
  • @TobiOetiker - Had the same thing again with version 1.7.1 do you want me to raise a bug? – user3788685 Nov 06 '21 at 20:17
  • sure :) ideally without php in the mix! – Tobi Oetiker Nov 08 '21 at 08:43
  • I'll see what I can do. I did notice that grafana exhibited the same problem as well. – user3788685 Nov 08 '21 at 11:20

0 Answers0