1

Is it possible for a bucket in cloud storage to receive data/messages from pubSub? if yes then how??

Currently i am publishing messages to pubsub and i want to use pull delivery type (for that i have to provide endpoint URL for the bucket, which i couldn't find anywhere)

I found this somewhere in there docs

But it didn't work.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
khrystal
  • 110
  • 2
  • 11

2 Answers2

2

No, sorry. GCS only accepts uploads of complete files via HTTP. You could build a small app that took incoming Pub/Sub messages and uploading them as separate GCS objects or batch them into groups of messages and upload those to GCS, but there's no such built-in functionality.

Can I ask you more about your use case? What are you trying to do?

Brandon Yarbrough
  • 37,021
  • 23
  • 116
  • 145
  • old question, but the basic idea is that I need instant logs from cloudsql. Sink to bucket only works on per hour basis. Pub sub is instant. I read my k8s cluster logs from the bucket already. It would make sense to redirect pubsub logs to the same bucket as well. So in this case, it is just a crutch to retrieve the logs from databases in the timely manner. – user3081519 Aug 29 '17 at 18:56
  • 1
    @user3081519 you can use Cloud Dataflow to read from PubSub and Write to GCS – vdolez Nov 21 '17 at 15:17
1

Pubsub subscription for GCS is now supported. You no longer need any dataflow or computing resources to transfer the data to GCS. enter image description here

This subscription also offers different options

  1. file format
  2. file name prefix/suffix
  3. batch duration
  4. maximum size per file
Balasubramanian Naagarajan
  • 338
  • 3
  • 17