The set up is this:
I've got a topic set up to receive messages, and then I've got a Dataflow pipeline set up using the Pub/Sub to GCS Text Template that dumps the messages into windowed text files in a GCS Bucket. This works fine--I eventually get files in GCS containing the test messages I send through the Console using the "Publish Message" button on my topic (side note I thought it would save the "data" portion of the message but it looks like it only saves the message body. But I can work around that).
The problem:
I'm intending to send messages to the topic using the python client (eventually from App Engine)...but I can't get it to work when I send them from my local machine. I am following the very simple examples in https://cloud.google.com/pubsub/docs/publisher, using the pubsub_v1 module. Even with the batch settings set to 1kb/1s I can never get any to publish successfully. I keep getting this error after 5 minutes of the future not responding:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/path/to/environment/lib/python2.7/site-packages/google/cloud/pubsub_v1/publisher/batch/thread.py", line 239, in monitor
return self._commit()
File "/path/to/environment/lib/python2.7/site-packages/google/cloud/pubsub_v1/publisher/batch/thread.py", line 204, in _commit
self._messages,
File "/path/to/environment/lib/python2.7/site-packages/google/cloud/pubsub_v1/gapic/publisher_client.py", line 325, in publish
return self._publish(request, retry=retry, timeout=timeout)
File "/path/to/environment/lib/python2.7/site-packages/google/api_core/gapic_v1/method.py", line 139, in __call__
return wrapped_func(*args, **kwargs)
File "/path/to/environment/lib/python2.7/site-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
on_error=on_error,
File "/path/to/environment/lib/python2.7/site-packages/google/api_core/retry.py", line 195, in retry_target
last_exc)
File "/path/to/environment/lib/python2.7/site-packages/six.py", line 737, in raise_from
raise value
RetryError: Deadline of 600.0s exceeded while calling <functools.partial object at 0x10662de68>, last exception: 503 Getting metadata from plugin failed with error: ('invalid_grant: Bad Request', u'{\n "error" : "invalid_grant",\n "error_description" : "Bad Request"\n}')
Code Sample:
>>> from google.cloud import pubsub_v1
>>> BATCH_SETTINGS = pubsub_v1.types.BatchSettings(max_bytes=1024,max_latency=1)
>>> publisher = pubsub_v1.PublisherClient(BATCH_SETTINGS)
>>> topic_path = publisher.topic_path("my-project", "topic-name")
>>> publisher.publish(topic_path, b'first message from python 3:38:59')
Update: After deploying my app everything works fine while in production. This is fine for my needs, but it still would be nice to know why it doesn't work while running locally in debug mode on my computer.
