1

We are building a data pipeline on GCP and chosen Bigtable as our data warehouse for raw events. My problem is - I cannot find a way to properly monitor the cluster. The dashboard offers a limited amount of metrics which could do the trick if there was any way to connect them into StackDriver without too much hassle, just like all the other services on GCP are set up.

Is it possible to monitor Bigtable with Stackdriver? If not, is there another recommended way to achieve this?

I found this post that explains how to get reports on the requests running against Bigtable, but this doesn't help me when trying to receive the big picture about the entire cluster's health.

Misha Brukman
  • 768
  • 8
  • 22
Yaron Idan
  • 187
  • 1
  • 6

2 Answers2

1

Google Cloud Bigtable exports server-side metrics to Stackdriver; you can find more information in the Bigtable documentation as well as Stackdriver metrics list.

In addition, if you are using the HBase-compatible Java client for Bigtable, you can export Dropwizard metrics on the client side which will give you detailed client-side performance metrics, such as end-to-end request latency, that you can track on your own, e.g., via your own Graphite instance.

These metrics are not meant to be a substitute to Stackdriver metrics for Bigtable, but to complement those metrics, and we would recommend tracking client-side metrics in addition to looking at server-side metrics.

Misha Brukman
  • 768
  • 8
  • 22
1

Note that it is now possible to query Bigtable metrics via the Stackdriver Monitoring API: https://cloud.google.com/monitoring/api/metrics#gcp-bigtable

You can also create alerts and define custom dashboards of Google Cloud Bigtable metrics via the Stackdriver UI: https://app.google.stackdriver.com/

Ramesh Dharan
  • 111
  • 4