0

I've got a dataflow job that pulls messages off of several Google Pub/Sub topics, does some parallel processing on the individual elements contained within those messages, then passes the collection on for further consumption by various resources. I'd like to put together a Stackdriver dashboard showing how many individual elements have been processed for each topic. Each ParDo step outputs a PCollection.

I've set up a dashboard using ElementCount, but I'm only able to filter by job, not by step. If I mouseover the line in the chart produced using ElementCount, I can see counts for every single step. Indeed, it does appear that the metrics for these are being reported, as I can use the gcloud commandline utility in the following manner:

gcloud beta dataflow metrics list [jobid] --filter ElementCount

...
name:
  context:
    original_name: extract_value_topic_1/Map-out0-ElementCount
    output_user_name: extract_value_topic_1/Map-out0
  name: ElementCount
  origin: dataflow/v1b3
scalar: 7000
updateTime: '2017-05-03T18:13:22.804Z'
---
name:
  context:
    original_name: extract_value_topic_2/Map-out0-ElementCount
    output_user_name: extract_value_topic_2/Map-out0
  name: ElementCount
  origin: dataflow/v1b3
scalar: 12000
updateTime: '2017-05-03T18:13:22.804Z'

I have several of these, but I don't see a straightforward way of building Stackdriver charts based on them (aside from logging to the console for every element processed then using that to generate a log-based metric, but that seems like it'd be extremely inefficient on a number of levels.) Am I missing something? How would one create a chart based on these ElementCounts?

Edit: Additionally, if I open up the Metrics Explorer I can enter dataflow/job/element_count into the search box then pcollection into the filter box, but I'm unable to build a dashboard with this chart in it as the filter selection in the dashboard chart builder does not allow for filtering by pcollection.

Gideon
  • 355
  • 3
  • 12

1 Answers1

0

Unfortunately, you currently cannot build a dashboard with a filter on a metric label. As you noticed, the new (Beta) Metric Explorer provides the filtering functionality and the Stackdriver team is actively working on providing that functionality to the dashboard charts as well. I will follow up if I receive any further updates or details from the Stackdriver team. --Andrea

Andrea
  • 191
  • 3
  • That's unfortunate, although this isn't a critical piece of our reporting functionality; it's just something that would tickle the higher-ups and something we could put on display in the lobby. In the meantime, are you aware of any workarounds using custom metrics or something similar? – Gideon May 04 '17 at 16:38
  • The Stackdriver API: https://developers.google.com/apis-explorer/#p/monitoring/v3/ does allow you to filter on labels. So you certainly could define a custom metric that combined the label and the metric name to produce a unique metric name. But I don't think I'd recommend that. I'm not sure what your timeframe is, but I did get this sense this functionality will be available in Stackdriver in the not too distant future. – Andrea May 05 '17 at 18:38