In the pipeline, I'm reading from Pub/Sub and I'm attempting to write to spanner. Writing to BigTable works, but spanner is a better option for my needs.
In the image below I've expanded the relevant steps. In the top right corner is the "Debug spanner" step, which shows the proper messages via LOG. I'm quite confident that SpannerIO.Write sees those messages as well.
Even without this debugging step, the messages still don't get written to spanner.
There is no exception whatsoever, but as you can see further down, CreateDataflowView is "not started", and it occurs twice in the pipeline.
Reading from pubsub and writing to spanner with less steps works - I've used a similar code from the pipeline displayed below to test.
What could cause this?
(the image is stiched together, but it contains the entire subtree until the PDone step)
The spanner steps are created with this code:
SpannerConfig spannerConfig = SpannerConfig.create()
.withProjectId("X")
.withInstanceId("X")
.withDatabaseId("X")
;
//spannerConfig.validate();//does not throw exception
//SpannerAccessor accessor = spannerConfig.connectToSpanner();//does not throw exception
PDone writtenToSpanner = encodedForSpanner.apply("write to spanner",
SpannerIO.write()
.withSpannerConfig(spannerConfig)
.withBatchSizeBytes(0)
);
