0

Is there a way we can monitor bolt specific metrics in ambari? Here is what i have tried. I was able to publish a sample metric with the help of https://github.com/bbende/ambari-metric-producer/blob/master/src/main/java/org/apache/ambari/MetricProducer.java Can we see this metric in ambari?

In HDFS metrics, we can add widget for HDFS/Hbase metrics for DataNode/NameNode which contain all the default metrics. Is it possible to have some custom metrics in this list such as a counter in one of the bolt in topology.

user2147688
  • 95
  • 1
  • 9

1 Answers1

2

To be able to use metrics in Ambari Widgets you must tell ambari about them by adding them to a metrics.json file. Each service managed by ambari has a corresponding metrics.json file. For instance here is the metrics.json file for the HBase service.

Determine which service is responsible for the metric you wish to add then modify that services metrics.json file to include it. You then must restart ambari-server for the changes to be picked up. You can then use the metric in any widget you desire.

Another configuration file that Ambari uses is the widgets.json file. It sits in the same directory as the metrics.json file. Some services do not have a widgets.json file defined and thus will not be able to add widgets using the services dashboard. Storm is one of these services. You can define a widgets.json for Storm and add it to the service. See the Ambari documentation on Widget Operations for more details.

cjackson
  • 1,637
  • 1
  • 13
  • 25
  • Thanks @cjackson. I want to monitor a few counters in my storm topologies. Also i see the add Widget option only for HDFS/HBASE services and not in STORM service. – user2147688 Mar 30 '16 at 14:18
  • I've amended my answer to explain why the STORM service doesn't have the add Widget button. – cjackson Mar 30 '16 at 15:18
  • Thanks. i see that change comes into effect if i change the metrics.json on ambari-server and the on server hosting the server as well under ambari-agent directory and restart the server. Is there any documentation on managing these files and how these files are copied from ambari-server to corresponding ambari-agent? – user2147688 Mar 31 '16 at 17:46
  • You should only need to modify those files at their source location which is on the ambari-server node at /var/lib/ambari-server/resources/common-services// or /var/lib/ambari-server/resources/stacks///services/. Once you restart ambari-server these files are copied to each node in the cluster via the ambari-agent. – cjackson Mar 31 '16 at 17:51
  • Thanks.realized it just after my previous comment. earlier i was updating metrics.json under common-services. I see the changes when file under /stacks/ is updated. Is there any documentation available on how to manage these files? – user2147688 Mar 31 '16 at 18:05
  • I meant the documentation mentioning when and how do we update these files. as you provided in your previous comment. Some elaborate document if it is available. – user2147688 Apr 01 '16 at 15:28
  • There is no such documentation that I know of. This is knowledge I've gathered by writing my own custom services, looking at the source code for the core Ambari Hadoop services and by asking questions on the ambari-user email list. The wiki which I've already linked to has some good information but is not complete. – cjackson Apr 01 '16 at 15:31