Stack Overflow
Stack Overflow

Questions tagged [google-cloud-pubsublite]

For programming/API questions related to Google Cloud Pub/Sub Lite, which provides asynchronous messaging and ingestion services for event-driven systems and streaming analytics on the Google Cloud Platform infrastructure at low cost of operation with pre-provisioned capacity.

29 questions
4
votes
0 answers

gcp pubsub-lite subscription python "A second subscriber connected with the same SubscriberId and topic partition" error

I am using the python pubsublite client(async version) for subscribing from pubsub-lite. The topic has 10 partitions, I am creating 5 clients at the same time so each client gets messages from two partitions(its working), but sometimes I am getting…
Vasu Kandimalla
  • 120
  • 8
3
votes
0 answers

Using pubsub lite library in spark getting error

I am getting error while publishing message to gcp pubsub lite using spark structured streaming. I cannot use writestream as I want to use it in forEachBatch sink in spark so I am using foreachpartition and foreach and publishing message inside…
Pranjal Singh
  • 31
  • 1
2
votes
1 answer

BigQuery streaming insert from Dataflow - no results

I have a Dataflow pipeline which is reading messages from PubSub Lite and streams data into a BigQuery table. The table is partitioned by day. When querying the table with: SELECT * FROM `my-project.my-dataset.my-table` WHERE DATE(timestamp) =…
p13rr0m
  • 1,107
  • 9
  • 21
1
vote
1 answer

Accessing pubsublite message attributes in beam pipeline - Java

We have been using PubSubLite in our Go program without any issues and I just started using the Java library with Beam. Using the PubSubLite IO, we get PCollection of SequencedMessage specifically:…
psykidellic
  • 71
  • 1
1
vote
1 answer

Google Pub Sub/Lite vs RabbitMQ

I would like to confront my understanding of google pubSub/lite vs RabbitMQ (using MQTT over WSS). My use case is that I need something like a topic exchange. To send messages individually or to all or to some. Having RabbitMQ I understand that I…
petr
  • 27
  • 1
  • 4
1
vote
2 answers

Pubsublite message acknowledgement not working

I'm using Google pubsublite. Small dummy topic with single partition and a few messages. Python client lib. Doing the standard SubscriberCluent.subscribe with callback. The callback places message in a queue. When the msg is taken out of the queue…
zpz
  • 354
  • 1
  • 3
  • 16
1
vote
2 answers

Google Cloud PubSub Lite subscription in C#

I'm writing a software with C# language and I need to create a subscription to a Pub/Sub Lite in Google Cloud. I understood that I can't user the Google.Cloud.PubSub.V1 nuget package because it doesn't work with the Lite version. I found…
TeoVr81
  • 989
  • 15
  • 33
1
vote
0 answers

Equally distributed data based on partitioning in Pub sub lite

i am new to GCP Have created 2 partition in Lite Topics . I am trying to publish 8 files which should equally distributed on those 2 partition. Is it possible to make this in Python? So far have refer to this…
abhi
  • 41
  • 1
  • 7
1
vote
1 answer

PubSub Lite: Acknowledge all messages currently in backlog

How can I acknowledge all messages that are currently in the backlog for Google PubSub Lite subscription. I tried using gcloud pubsub lite-subscriptions ack-up-to SUBSCRIPTION \ --location LOCATION --partition PARTITION --offset OFFSET --project…
p13rr0m
  • 1,107
  • 9
  • 21
0
votes
0 answers

Cumulative Sum of Pubsub Unacked Messages

I want to get the TOTAL number of messages received by a subscription during a 3 day time period. However, I cannot just use the view of unacked messages as during this 3 day period as many messages were ack'ed and processed. I want to see the total…
Shaurya Sanghvi
  • 1
0
votes
0 answers

Error creating the Regional terrafom module of pubsub lite (PubSub Lite Topic Location)

While creating a terraform module PubSub Topic Lite there should be two options for location: Regional or Zonal. Actual Behavior For deploying Zonal Topic, it is straight forward since you only need to provide a region and a zone. (No issue) For…
Jaison K J
  • 1
  • 1
0
votes
0 answers

Is there a way to monitor a gmail inbox for changes?

I'm looking for a way to monitor a gmail inbox and alert if a specific email is not received by end of day. the account is not user-owned (its a business email dedicated to this purpose) So I need a way to make this work without being "logged in"…
smansou
  • 1
  • 1
0
votes
0 answers

How to write tests with Google's PubSubLite?

With Google's PubSub there is an embedded broker available for writing tests: embedded-google-pubsub (https://github.com/PlaytikaOSS/testcontainers-spring-boot/blob/develop/embedded-google-pubsub/README.adoc) Can I use this also for PubSubLite?…
micgn
  • 245
  • 1
  • 4
  • 14
0
votes
0 answers

Apache Nifi Not Publishing to PubsubLite Topics using Google Service Account

We are able to Publish events from Apache Nifi to Google Pubsub Lite using Application Default Credentials created using gcloud auth application-default login We want to use Google Service account instead of ADC but we are getting the below…
Raju Penki
  • 1
0
votes
0 answers

PublishGCPubSubLite processor in Apache NiFi is not working as expected

We are Using Apache Nifi to Publish events to PubsubLite using (PublishGCPubSubLite 1.17.0) Nifi processor. GCP SA json files contains below field: { "type": "service_account", "project_id": "", "private_key_id": "", "private_key":…
priyanka
  • 27
  • 4
1
2