According to this announcement by the Google Dataflow team, we should be able to see the details of our BigQuery sources and sinks in the console if we use the 1.6 SDK.
However, although the new "Pipeline Options" do indeed show up, the details of the BigQuery source/sinks do not display. Our pipeline is running in Batch mode. Job id for reference is: 2016-06-23_04_38_00-7749718775594891108
How do we get the details of the BigQuery source/sinks to show up?
I'm one of the dataflow developers who added this feature to the dataflow UI.
Currently these details are only visible when you click on the expanded steps. If click the arrow icon to fully expand your sources/sinks and click on the steps that show up after its expanded then you can see details that were specified for that step such as the BigQuery Query or table information.
Is there a particular detail/parameter you are looking for and not seeing for a step in your pipeline?
