2

I'm using the mysql-cacti-templates to monitor my servers. They include a set of MongoDB templates and tools.

So, I've got these running to monitor MongoDB connections. My poller reports:

02/16/2012 06:20:58 PM - CMDPHP: Poller[0] Host[26] DS[188] CMD: /usr/bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host db.live --type mongodb --items dc , output: dc:1

With

output: dc:1

being the value I'm after - 1 connection.

I'm totally befuddled however in that the graphs show up with a current value of 3524!

I've checked all the bindings between the RRD files, the mapping from 'dc' to the graph etc. etc. but remain baffled as to where the hell it's getting 3524 from.

I've looked in the RRD files using the dump tool and althought I don't really understand the output, the value of 3524 is clearly visible.

Where can I look in the chain of processing to get a better handle on where this value is appearing from?

Update:

Checked the cacti log file and I see this:

02/16/2012 03:32:29 PM - CMDPHP: Poller[0] Host[23] DS[171] CMD: /usr/bin/php -q /usr/share/cacti/scripts/ss_get_by_ssh.php --host spider.live --type mongodb --items dc , output: dc:1
02/16/2012 03:32:29 PM - POLLER: Poller[0] CACTI2RRD: /usr/bin/rrdtool update /usr/share/cacti/rra/spider_live_mongodb_cnctd_clnts_171.rrd --template MONGODB_cnctd_clnts 1329406349:3521

How on earth is it translating 1 into 1329406349:3521 ?

Ted
  • 143
  • 6
  • http://docs.cacti.net/manual:087:4_help.2_debugging – brandeded Feb 16 '12 at 18:32
  • Yep, been through all that. – Ted Feb 16 '12 at 20:11
  • Can you provide an example line from your RRA dump? – thinice Feb 16 '12 at 20:58
  • Understanding how RRD stores data is difficult to get into - but you may find this useful as you continue troubleshooting: http://oss.oetiker.ch/rrdtool/tut/rrd-beginners.en.html – thinice Feb 16 '12 at 20:59
  • Dave, reading the above log, did you look at the script `/usr/share/cacti/scripts/ss_get_by_ssh.php` and seeing what it does with the given input of `dc:1`? It appears the this is calling `/usr/bin/rrdtool update` aka `CACTI2RRD`, right? – brandeded Feb 16 '12 at 21:47
  • That's not my understanding. The ...get_by_ssh.php script retrieves the value which the poller then uses to call the rrdtool to update the RR archives. As far as I can see the poller is using a different value than the one retrieved. – Ted Feb 16 '12 at 22:10
  • What is `the poller` and how does it call `rrdtool`? I can't find this information. – brandeded Feb 16 '12 at 22:30
  • Also `Check MySQL Update In most cases, this step make be skipped. You may want to return to this step, if the next one fails (e.g. no rrdtool update to be found) From debug log, please find the MySQL update statement for that host concerning table poller_output. On very rare occasions, this will fail. So please copy that sql statement and paste it to a mysql session started from cli. This may as well be done from some tool like phpMyAdmin. Check the sql return code.` ??? – brandeded Feb 16 '12 at 22:35
  • I grapped the cacti source for `poller_output`, and think I located the calls in `cmd.php` and `poller.php`. the function `db_execute()`, which is declared in `database.php`, seems to be called to insert the poller_output. You should be able to debug your problem from one of those two files. within the db_execute(), try searching the `$sql` string for the string `poller_output`. if so, log the value, and try this from that low level all the way to the caller php. – brandeded Feb 16 '12 at 22:58

1 Answers1

2

Well after wasting several hours of my life I nailed it.

http://bugs.cacti.net/view.php?id=2063

"dc:1" is being is being evaluated as hexadecimal because 'dc' is 'hex'.

Crazy bit of code...

Ted
  • 143
  • 6
  • Sounds like it's being confused by the `output:`. Cacti expects `field1:value1 field2:value2 ...` and is seeing a field called `output` with a value of `dc:1`. – pgs Mar 28 '12 at 00:00