1

I'm trying to test my DataFlow job locally, but keep getting timeout errors on reading from PubSub topic.

PCollection<KV<Integer, Tick>> input = pipeline.apply(PubsubIO.Read.topic(options.getPubSubTopic()).maxNumRecords(10))

I'm sure that topic name is correct since the DataFlow is correctly creating subscription under desired topic each time I run the job and I can see it in the browser.

Any hints what might be the issue here?

Adrian Serafin
  • 7,665
  • 5
  • 46
  • 67
  • Could you post more details about the exception you're getting? – Ben Chambers Aug 10 '16 at 17:48
  • @BenChambers it turns out that it was bc. topic was empty - I was sure that's not the case since I was checking API Manager and I could see call count for Pub/Sub increasing with Error 0% and in fact I was getting http code 400 when publishing a message :( – Adrian Serafin Aug 10 '16 at 19:12

1 Answers1

0

The issue was that the topic was empty. Even though there were attempted publish calls to the topic, they were failing with 400s, so nothing made it to the pipeline's subscription.

Sam McVeety
  • 3,194
  • 1
  • 15
  • 38