Gumbo Apache Storm Monitoring Metrics

Question

I am trying to configure Gumbo for storm topology monitoring from here

There is not clear example or usage given on the site, need some clarification on what are the parameter and where to add this code given on the site above

MonitorClient mclient = MonitorClient.forConfig(conf);

// There are multiple metric groups, each with multiple metrics.
// Components have names and multiple instances, each of which has an    integer ID

mclient.declare(metricGroup,metric,task_id,component_id);

mclient.increment(metricGroup,metric, 1L , task_id);

TaskHook.registerTo(config);

Now what values we need to provide for MetricGroup, metric, task_id and component_id? If need to find it from each Spout and Bolt how can we do it? Where should this code be placed, is it in topology builder before submitting the topology or in individual Spout/Bolt class under open/prepare methods or somewhere else. Appreciate any help on this question.

score 0 · Accepted Answer · answered Mar 28 '16 at 01:25

I tried few option and below is the config that worked for me, the Group Name can be anything, Metric name is the name of the stream going out from one component to other, taskid can be any unique task number,

conf.put("gumbo.server.kind", "local");
conf.put("gumbo.local.port", 8086); //Any port it must be same in the html file
conf.put("gumbo.start", System.currentTimeMillis());  // should be the same for all calls
conf.put("gumbo.bucketSize", 1000L);
conf.put("gumbo.enabled", true);
conf.put("gumbo.http.host", "hostname");
conf.put("gumbo.http.port", 8086);//Any port it must be same in the html file
conf.put("gumbo.http.app", "gumbo");
conf.put("gumbo.enabled", true);
conf.put("gumbo.server.key", topology_id);

MonitorClient mclient = MonitorClient.connect(conf);

GumboTaskHook.registerTo(conf);
    mclient.declare("Backlog",RTConstants.MATCH_LEFT_STREAM,3,RTConstants.TRANSFORM_LEFT_BOLT);       
    mclient.increment("Backlog",RTConstants.MATCH_LEFT_STREAM, 1L , 3);

    mclient.declare("Backlog",RTConstants.MATCH_RIGHT_STREAM,4,RTConstants.TRANSFORM_RIGHT_BOLT);       
    mclient.increment("Backlog",RTConstants.MATCH_RIGHT_STREAM, 1L , 4);

Gumbo Apache Storm Monitoring Metrics

1 Answers1