0

I'm new to GCP. I have created a Pub/Sub Topic and a Cloud Function. 1.When a Message is Published to Topic the Cloud function is Executed and it invokes a DAG in Cloud Composer.

I want to read the published message in the DAG file. how to do that?

Akshay
  • 25
  • 1
  • 7
  • 1
    Can you check this answer, as, from my point of view, it may help: [Trigger Cloud Composer DAG with a Pub/Sub message](https://stackoverflow.com/a/58575463/6073141) – al-dann May 06 '21 at 11:24
  • @al-dann am able to trigger the DAG. I facing an issue to pull pub/sub message into a DAG as a variable. – Akshay May 06 '21 at 14:53
  • I meant this link from the answer I mentioned above: https://incubator-airflow.readthedocs.io/en/latest/howto/operator/google/cloud/pubsub.html#pulling-messages-from-a-pubsub-subscription – al-dann May 06 '21 at 19:22
  • @al-dann I used another approach. 1) I have created a table in Bigquery. 2). I used Bigquery Client in cloud function which adds records to the Bq table and then it triggers the DAG. 3) In that I used a BigQueryHook to read data from Bq. – Akshay Jun 02 '21 at 13:26
  • @al-dann Here the Link of i refered for bigQuery Hook https://stackoverflow.com/questions/53565834/fetch-results-from-bigqueryoperator-in-airflow – Akshay Jun 02 '21 at 13:27

0 Answers0