how to get a total bandwidth graph in munin

Question

I have munin-node in a basic default configuration on all my servers, I would like to create a summary chart that displays total bandwidth in/out http and another chart showing total bandwidth in/out for all the interfaces.

[Edit] After following the first poster suggestions, I am currently up to here;

I am trying to achieve a aggregate SUM graph of up/down for if_eth0 service

based on the details in the FAQ; http://munin-monitoring.org/wiki/faq#Q:HowdoIusefieldname.sum

and the detailed example; http://munin-monitoring.org/wiki/aggregate_examples

and this interesting post on aggregates; http://blog.loftninjas.org/2010/04/08/an-evening-with-munin-graph-aggregation/

I have the following to define my hosts in /etc/munin/munin.conf;

    [cirrusj1605.hpl.com]
    address 19x.xx.xx.205
    use_node_name yes
    [cirrusj1606.hpl.com]
    address 19x.xx.xx.206
    use_node_name yes

...

and an aggregates virtual host like so;

[hpl.com;aggregates] update no contacts no

    total_bandwidth.graph_title Aggregated bandwidth
    total_bandwidth.upload.label upload
    total_bandwidth.upload.sum \
    cirrusj1605.hpl.com:if_eth0:up \
    cirrusj1606.hpl.com:if_eth0:up

If I try to run the munin-graph manually I get the following error when producing each of the graphs;

2011/02/24 20:04:39 [RRD ERROR] Unable to graph /var/www/html/munin/hpl.com/aggregates/total_bandwidth-month.png : invalid rpn expression in: az1_0,UN,0,az1_0,IF,,+

full trace follows, any suggestion on what the problem might be?

Thanks, T

> -bash-3.2$ /usr/share/munin/munin-graph --debug
> --nofork --nolazy --host aggregates 2011/02/24 20:04:39 [DEBUG] Creating
> new lock file
> /var/run/munin/munin-graph.lock
> 2011/02/24 20:04:39 [DEBUG] Creating
> lock : /var/run/munin/munin-graph.lock
> succeeded 2011/02/24 20:04:39 Starting
> munin-graph 2011/02/24 20:04:39
> [DEBUG] Doing work synchrnonously
> 2011/02/24 20:04:39 [DEBUG] Node name:
> total_bandwidth 2011/02/24 20:04:39
> [DEBUG] Expanding specials for
> total_bandwidth: "upload". 2011/02/24
> 20:04:39 DEBUG: expand_specials
> (cirrusj1605.hpl.com:if_eth0:up):
> Doing sum... 2011/02/24 20:04:39
> [DEBUG] Checking field lengths for
> total_bandwidth:
> "z1_0=cirrusj1605.hpl.com:if_eth0:up".
> 2011/02/24 20:04:39 [DEBUG] Treating
> fields
> "z1_0=cirrusj1605.hpl.com:if_eth0:up","upload".
> 2011/02/24 20:04:39 [DEBUG] Processing
> field "z1_0" [z1_0]. 2011/02/24
> 20:04:39 [PERL WARNING] Use of
> uninitialized value in subroutine
> entry at
> /usr/lib/perl5/vendor_perl/5.8.8/Munin/Master/GraphOld.pm
> line 757. 2011/02/24 20:04:39 DEBUG:
> single_value: Checking field "z1_0".
> 2011/02/24 20:04:39 DEBUG:
> single_value: Checking field "upload".
> 2011/02/24 20:04:39 [DEBUG] service
> hpl.com :: aggregates ::
> total_bandwidth has 2 elements.
> 2011/02/24 20:04:39 [PERL WARNING] Use
> of uninitialized value in
> concatenation (.) or string at
> /usr/lib/perl5/vendor_perl/5.8.8/Munin/Master/GraphOld.pm
> line 774. 2011/02/24 20:04:39 [DEBUG]
> RRD name / filename: z1_0 / 2011/02/24
> 20:04:39 [PERL WARNING] Use of
> uninitialized value in concatenation
> (.) or string at
> /usr/lib/perl5/vendor_perl/5.8.8/Munin/Master/GraphOld.pm
> line 783. 2011/02/24 20:04:39 [PERL
> WARNING] Use of uninitialized value in
> concatenation (.) or string at
> /usr/lib/perl5/vendor_perl/5.8.8/Munin/Master/GraphOld.pm
> line 785. 2011/02/24 20:04:39 [PERL
> WARNING] Use of uninitialized value in
> concatenation (.) or string at
> /usr/lib/perl5/vendor_perl/5.8.8/Munin/Master/GraphOld.pm
> line 786. 2011/02/24 20:04:39 [DEBUG]
> Field name after cdef set to cdefz1_0
> 2011/02/24 20:04:39 [DEBUG] Drawing
> field "z1_0". 2011/02/24 20:04:39
> 
> rrdtool 'graph' '--font' \
>     'LEGEND:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '--font' \
>     'UNIT:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '--font' \
>     'AXIS:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '-W' \
>     'Munin 1.4.5' \
>     '/var/www/html/munin/hpl.com/aggregates/total_bandwidth-month.png'
> \
>     '--title' \
>     'Aggregated bandwidth - by month' \
>     '--start' \
>     '-33d' \
>     '--height' \
>     '175' \
>     '--width' \
>     '400' \
>     '--imgformat' \
>     'PNG' \
>     'DEF:az1_0=:42:MAX' \
>     'DEF:iz1_0=:42:MIN' \
>     'DEF:gz1_0=:42:AVERAGE' \
>     'CDEF:acdefz1_0=az1_0,UN,0,az1_0,IF,,+'
> \
>     'CDEF:icdefz1_0=iz1_0,UN,0,iz1_0,IF,,+'
> \
>     'CDEF:gcdefz1_0=gz1_0,UN,0,gz1_0,IF,,+'
> \
>     'CDEF:ccdefz1_0=gcdefz1_0' \
>     'COMMENT:      ' \
>     'COMMENT: Cur\:' \
>     'COMMENT:Min\:' \
>     'COMMENT:Avg\:' \
>     'COMMENT:Max\:  \j' \
>     'LINE1.6:gcdefz1_0#00CC00:upload ' \
>     'GPRINT:ccdefz1_0:LAST:%6.2lf%s' \
>     'GPRINT:icdefz1_0:MIN:%6.2lf%s' \
>     'GPRINT:gcdefz1_0:AVERAGE:%6.2lf%s' \
>     'GPRINT:acdefz1_0:MAX:%6.2lf%s\j' \
>     'COMMENT:Last update\: Wed Dec 31 16\:00\:00 1969\r' 2011/02/24 20:04:39
> [RRD ERROR] Unable to graph
> /var/www/html/munin/hpl.com/aggregates/total_bandwidth-month.png
> : invalid rpn expression in:
> az1_0,UN,0,az1_0,IF,,+ 2011/02/24
> 20:04:39
> 
> rrdtool 'graph' '--font' \
>     'LEGEND:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '--font' \
>     'UNIT:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '--font' \
>     'AXIS:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '-W' \
>     'Munin 1.4.5' \
>     '/var/www/html/munin/hpl.com/aggregates/total_bandwidth-week.png'
> \
>     '--title' \
>     'Aggregated bandwidth - by week' \
>     '--start' \
>     '-8d' \
>     '--height' \
>     '175' \
>     '--width' \
>     '400' \
>     '--imgformat' \
>     'PNG' \
>     'DEF:az1_0=:42:MAX' \
>     'DEF:iz1_0=:42:MIN' \
>     'DEF:gz1_0=:42:AVERAGE' \
>     'CDEF:acdefz1_0=az1_0,UN,0,az1_0,IF,,+'
> \
>     'CDEF:icdefz1_0=iz1_0,UN,0,iz1_0,IF,,+'
> \
>     'CDEF:gcdefz1_0=gz1_0,UN,0,gz1_0,IF,,+'
> \
>     'CDEF:ccdefz1_0=gcdefz1_0' \
>     'COMMENT:      ' \
>     'COMMENT: Cur\:' \
>     'COMMENT:Min\:' \
>     'COMMENT:Avg\:' \
>     'COMMENT:Max\:  \j' \
>     'LINE1.6:gcdefz1_0#00CC00:upload ' \
>     'GPRINT:ccdefz1_0:LAST:%6.2lf%s' \
>     'GPRINT:icdefz1_0:MIN:%6.2lf%s' \
>     'GPRINT:gcdefz1_0:AVERAGE:%6.2lf%s' \
>     'GPRINT:acdefz1_0:MAX:%6.2lf%s\j' \
>     'COMMENT:Last update\: Wed Dec 31 16\:00\:00 1969\r' 2011/02/24 20:04:39
> [RRD ERROR] Unable to graph
> /var/www/html/munin/hpl.com/aggregates/total_bandwidth-week.png
> : invalid rpn expression in:
> az1_0,UN,0,az1_0,IF,,+ 2011/02/24
> 20:04:39
> 
> rrdtool 'graph' '--font' \
>     'LEGEND:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '--font' \
>     'UNIT:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '--font' \
>     'AXIS:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '-W' \
>     'Munin 1.4.5' \
>     '/var/www/html/munin/hpl.com/aggregates/total_bandwidth-day.png'
> \
>     '--title' \
>     'Aggregated bandwidth - by day' \
>     '--start' \
>     '-30h' \
>     '--height' \
>     '175' \
>     '--width' \
>     '400' \
>     '--imgformat' \
>     'PNG' \
>     'DEF:az1_0=:42:MAX' \
>     'DEF:iz1_0=:42:MIN' \
>     'DEF:gz1_0=:42:AVERAGE' \
>     'CDEF:acdefz1_0=az1_0,UN,0,az1_0,IF,,+'
> \
>     'CDEF:icdefz1_0=iz1_0,UN,0,iz1_0,IF,,+'
> \
>     'CDEF:gcdefz1_0=gz1_0,UN,0,gz1_0,IF,,+'
> \
>     'CDEF:ccdefz1_0=gcdefz1_0' \
>     'COMMENT:      ' \
>     'COMMENT: Cur\:' \
>     'COMMENT:Min\:' \
>     'COMMENT:Avg\:' \
>     'COMMENT:Max\:  \j' \
>     'LINE1.6:gcdefz1_0#00CC00:upload ' \
>     'GPRINT:ccdefz1_0:LAST:%6.2lf%s' \
>     'GPRINT:icdefz1_0:MIN:%6.2lf%s' \
>     'GPRINT:gcdefz1_0:AVERAGE:%6.2lf%s' \
>     'GPRINT:acdefz1_0:MAX:%6.2lf%s\j' \
>     'COMMENT:Last update\: Wed Dec 31 16\:00\:00 1969\r' 2011/02/24 20:04:39
> [RRD ERROR] Unable to graph
> /var/www/html/munin/hpl.com/aggregates/total_bandwidth-day.png
> : invalid rpn expression in:
> az1_0,UN,0,az1_0,IF,,+ 2011/02/24
> 20:04:39
> 
> rrdtool 'graph' '--font' \
>     'LEGEND:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '--font' \
>     'UNIT:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '--font' \
>     'AXIS:7:/usr/share/fonts/bitstream-vera/VeraMono.ttf'
> \
>     '-W' \
>     'Munin 1.4.5' \
>     '/var/www/html/munin/hpl.com/aggregates/total_bandwidth-year.png'
> \
>     '--title' \
>     'Aggregated bandwidth - by year' \
>     '--start' \
>     '-400d' \
>     '--height' \
>     '175' \
>     '--width' \
>     '400' \
>     '--imgformat' \
>     'PNG' \
>     'DEF:az1_0=:42:MAX' \
>     'DEF:iz1_0=:42:MIN' \
>     'DEF:gz1_0=:42:AVERAGE' \
>     'CDEF:acdefz1_0=az1_0,UN,0,az1_0,IF,,+'
> \
>     'CDEF:icdefz1_0=iz1_0,UN,0,iz1_0,IF,,+'
> \
>     'CDEF:gcdefz1_0=gz1_0,UN,0,gz1_0,IF,,+'
> \
>     'CDEF:ccdefz1_0=gcdefz1_0' \
>     'COMMENT:      ' \
>     'COMMENT: Cur\:' \
>     'COMMENT:Min\:' \
>     'COMMENT:Avg\:' \
>     'COMMENT:Max\:  \j' \
>     'LINE1.6:gcdefz1_0#00CC00:upload ' \
>     'GPRINT:ccdefz1_0:LAST:%6.2lf%s' \
>     'GPRINT:icdefz1_0:MIN:%6.2lf%s' \
>     'GPRINT:gcdefz1_0:AVERAGE:%6.2lf%s' \
>     'GPRINT:acdefz1_0:MAX:%6.2lf%s\j' \
>     'COMMENT:Last update\: Wed Dec 31 16\:00\:00 1969\r' 2011/02/24 20:04:39
> [RRD ERROR] Unable to graph
> /var/www/html/munin/hpl.com/aggregates/total_bandwidth-year.png
> : invalid rpn expression in:
> az1_0,UN,0,az1_0,IF,,+ 2011/02/24
> 20:04:39 Graphed service :
> total_bandwidth (0.01 sec * 4)
> 2011/02/24 20:04:39 Munin-graph
> finished (0.11 sec)

score 4 · Accepted Answer · edited Mar 02 '11 at 19:13

Yes, it is done on the server side by creating a combined graph.

Basically it is done by describing a new graph in munin server configuration (munin.conf) where the data graphed is the sum of two data sources :

[some.machine.boo]
    total_http_bw.graph_order total_received
    total_http_bw.graph_title HTTP traffic recived by machine1 and machine2
    total_http_bw.graph_vlabel Kb/s
    total_http_bw.total_received.label HTTP received traffic
    total_http_bw.total_received.sum \
            machine1.machine.boo:http_bw:received \
            machine2.machine.boo:http_bw:received

You can find an example of stacking (graphing two data sources one on the top of the other) on munin web site. If you want to graph the sum of two data sources you can use the total_received.sum function instead of total_received.stack.

[edit] ive added the stanzas required to have the net interface bandwidth to this answer, as it makes for a more complete answer to the original question (and I can't select 2 correct answers etc)

I had a lot of trouble getting the net interfaces to give up their values as a SUM, so here is the end result of lots of random trial and error changes to get a SUM line chart of multiple munin nodes

    [mydomain.com;aggregates]
    total_bandwidth.graph_args --base 1000 -l 0
    total_bandwidth.cdef 0
    total_bandwidth.graph_category Network
    total_bandwidth.graph_title Aggregated bandwidth
    total_bandwidth.graph_vlabel Bits/sec
    total_bandwidth.upload.label upload
    total_bandwidth.total.graph yes
    total_bandwidth.upload.sum \
    mybox1.mydomain.com:if_eth0.up \
    mybox2.mydomain.com:if_eth0.up
    total_bandwidth.upload.type COUNTER
    total_bandwidth.download.type COUNTER
    total_bandwidth.download.label download
    total_bandwidth.graph_order upload download
    total_bandwidth.total.graph no
    total_bandwidth.download.sum \
    mybox1.mydomain.com:if_eth0.down \
    mybox2.mydomain.com:if_eth0.down

I suspect a few of those directives are either unnecessary or plain wrong, but I hope it is a working starting point for anyone trying to achieve the same.

I also suspect that it works better if you explicitly make sure that all the hosts are in the aggregates group, eg mydomain.com even if your hosts are incorrectly named locally eg hostname -f is wrong

how to get a total bandwidth graph in munin

1 Answers1

Linked