0

I am trying to deploy an apache beam Data Flow pipeline built-in python in google cloud build. I don't find any specific details about constructing the cloud build.YAML file.

I found a link dataflow-ci-cd-with-cloudbuild, but this seems JAVA based, tried this too but did not work as my starting point is main.py

enter image description here

Vivek Ranjan
  • 1,432
  • 2
  • 15
  • 37
  • Do you want to run your dataflow from CloudBuild? Or to run a beam pipeline in Cloud Build runtime? Can you also share how you run today the dataflow, without cloud build? – guillaume blaquiere Feb 12 '21 at 15:17
  • The intent is to automate template creation. The above steps are taken from https://medium.com/@zhongchen/dataflow-ci-cd-with-cloudbuild-1ad503c1c81. Seems need own docker image to build this https://github.com/GoogleCloudPlatform/cloud-builders-community/tree/master/dataflow-python3 – Vivek Ranjan Feb 12 '21 at 15:19
  • How do you do today without Cloud Build? – guillaume blaquiere Feb 12 '21 at 16:20
  • It's done with cloud build using the yaml file from here https://medium.com/@zhongchen/dataflow-ci-cd-with-cloudbuild-1ad503c1c81 – Vivek Ranjan Feb 13 '21 at 14:58

1 Answers1

0

It requires a container registry. Step to build & deploy is explained in the below link

Github link

Vivek Ranjan
  • 1,432
  • 2
  • 15
  • 37