0

We have two different survey tools (Alice and Bob). Each tool is integrated as a data source with BigQuery and the data syncs into their own tables (surveys_by_alice and surveys_by_bob).

We want to update our current thinking in a special table based on the latest survey results (current_thinking).

What I have built so far:

  1. an on-demand scheduled query that reads from surveys_by_alice and surveys_by_bob and updates current_thinking
  2. a service account for this task with BigQuery Admin and Cloud Run Invoker permissions
  3. I went to Cloud Logging and created a sink based on when data gets inserted into the first survey table (called sink.surveys_by_alice) and a Pub/Sub topic (called topic.surveys_by_alice)
  4. I created a new cloud function with an Eventarc trigger based on the topic.surveys_by_alice Pub/Sub topic
  5. I deployed the code and it successfully triggers the on-demand scheduled query whenever the surveys_by_alice table is updated

Now, I also want the same cloud function to launch the same scheduled query whenever surveys_by_bob gets updated.

I am not sure what to do because I started by creating a second sink (sink.surveys_by_bob) and Pub/Sub topic (topic.surveys_by_bob), but I realize that there does not appear to be a way to edit my existing cloud function to also be triggered by topic.surveys_by_bob).

Cloud Functions settings with single Eventarc trigger

Am I missing something?

One solution is to simply have any updates to surveys_by_alice and surveys_by_bob point to the same Pub/Sub topic topic.surveys, but that seems like a problem if I want to differentiate the two for other downstream events.

TMo
  • 435
  • 4
  • 11

1 Answers1

1

As you said, one solution is to point both logging sinks to one single pub/sub topic which will trigger your cloud funtion.

If you want to keep the logging sinks and pub/sub topics separate but only one cloud funtion, you can:

  1. Change the the cloud function to be triggered by http request.
  2. Create the 2 pub/sub topics
  3. For each topic you create a push subscription pointing to the same url of the cloud function. Pub/sub Subscription

Last but not least, you can copy the cloud funtion and trigger the copy by the second topic (topic.surveys_by_bob). In this case you have 2 logging sinks, 2 pub/sub topics and 2 cloud functions

99_m4n
  • 1,239
  • 3
  • 17