Stack Overflow
Stack Overflow

Questions tagged [google-cloud-pubsub]

For programming/API questions related to Google Cloud Pub/Sub, which provides asynchronous messaging and ingestion services for event-driven systems and streaming analytics on the Google Cloud Platform infrastructure.

The official documentation can be found here: https://cloud.google.com/pubsub/docs/overview

2920 questions
8
votes
2 answers

How to route dead letter messages back to the original topic?

I have a Google Cloud Pub/Sub subscription that is using a dead-letter topic. I recently had an outage that was preventing a number of messages from being processed & they ended up in the dead-letter topic. The outage was resolved and I'd like to…
Mark McDonald
  • 7,571
  • 6
  • 46
  • 53
8
votes
2 answers

How to subscribe to multiple Google PubSub Projects in Spring GCP?

I want to subscribe to multiple Google Cloud PubSub projects in a Spring Boot application. After reading the related questions in How to wire/configure two pubsub gcp projects in one spring boot application with spring cloud?, How to use Spring…
Kabilesh
  • 1,000
  • 6
  • 22
  • 47
8
votes
2 answers

Message is received from Google Pub/Sub subscription again and again after acknowledge[Heisenbug]

I would like to notice that the scenarion I will describe happen rare enough and in most cases everything works as expected. I have 1 topic and 1 subscription on Pub/Sub side. My java application listens for subscription, does some processing and…
gstackoverflow
  • 36,709
  • 117
  • 359
  • 710
8
votes
0 answers

grpc StatusRuntimeException on Dataflow

I have a dataflow pipeline in which I consume pubsub messages, treat them, and then publish to pubsub. Whenever I have too many calculations (ie I increase the amount of treatment for each message) I get an Exception. :…
Jurgen Palsma
  • 300
  • 1
  • 3
  • 16
8
votes
1 answer

Google PubSub Pull vs Streaming Pull differences

I was reading the documentation from Google about pull vs streaming pull but could not quite get it. Can someone explain to me the difference and also would be helpful if someone has implemented it and could point me to the resources. Many thanks.
Shashesh
  • 131
  • 1
  • 4
8
votes
2 answers

Dataflow Template Cloud Pub/Sub Topic vs Subscription to BigQuery

I'm setting up a simple Proof of Concept to learn some of the concepts in Google Cloud, specifically PubSub and Dataflow. I have a PubSub topic greeting I've created a simple cloud function that sends publishes a message to that topic: const…
JonathanW
  • 131
  • 5
8
votes
2 answers

Google PubSub - serviceAccount:gmail-api-push@system.gserviceaccount.com doesn't exist

I am trying to subscribe to Gmail for mail notifications using Google's Pub/Sub and I've gone through the documentation and tutorials provided by Google. At one point, the docs state that I need to publish the correct rights to my PubSub topic: You…
Maxwell
  • 6,532
  • 4
  • 37
  • 55
8
votes
3 answers

Google PubSub and duplicated messages from the TOPIC

How to prevent duplicated msg from happening in Google Cloud PubSub? Say, I have a code that handles the msg that it is subscribed for. Say, I have 2 nodes with the same Service that has this code. Once one has received the msg but not yet…
ses
  • 13,174
  • 31
  • 123
  • 226
8
votes
2 answers

Batching PubSub requests

The NODEJS example code for batching pubsub requests looks like this: // Imports the Google Cloud client library const PubSub = require(`@google-cloud/pubsub`); // Creates a client const pubsub = new PubSub(); /** * TODO(developer): Uncomment the…
Erik van den Hoorn
  • 563
  • 5
  • 16
8
votes
1 answer

GCP PubSub: Publish message via CURL type of request

Does anyone have a working example of how to publish a message to a GCP PubSub topic via CURL type of commands, directly from shell? I'm trying to not use the CLI and not use the client libraries, and I'm getting hung up on the OAUTH stuff. I'd be…
Rich Murnane
  • 2,697
  • 1
  • 11
  • 23
8
votes
4 answers

Run synchronous pull in Google Cloud Pub/Sub with the Python client API

I can't find the returnImmediately flag in the Python client API. Is there any specific reason for that? Is there another way to pull queued message synchronously from a subscription in Python?
Rupert Schiessl
  • 799
  • 6
  • 11
8
votes
1 answer

How to invoke other Cloud Firebase Functions from a Cloud Function

Let's say I have a Cloud Firebase Function - called by a cron job - that produces 30+ tasks every time it's invoked. These tasks are quite slow (5 - 6 second each in average) and I can't process them directly in the original because it would time…
janesconference
  • 6,333
  • 8
  • 55
  • 73
8
votes
1 answer

java.lang.NoClassDefFoundError: com/google/common/base/MoreObjects when initializing pubsub on appengine

I try to run this code on appEngine (with java8 defined in my web.xml) public TopicName createTopic(final String topicNameStr) throws Exception { checkInit(); final TopicAdminSettings topicAdminSettings = …
Elad Benda2
  • 13,852
  • 29
  • 82
  • 157
8
votes
3 answers

Caused by: io.grpc.StatusRuntimeException: NOT_FOUND: Resource not found

I try to write a test of pubsub: @Test public void sendTopic() throws Exception { CustomSubscriber customSubscriber = new CustomSubscriber(); customSubscriber.startAndWait(); CustomPublisher customPublisher = new CustomPublisher(); …
Elad Benda
  • 35,076
  • 87
  • 265
  • 471
8
votes
2 answers

Stream BigQuery table into Google Pub/Sub

I have a Google bigQuery Table and I want to stream the entire table into pub-sub Topic what should be the easy/fast way to do it? Thank you in advance,
MoShe
  • 6,197
  • 17
  • 51
  • 77
1 2 3
…
99 100