0

I want to get the list of all pipelines run as a daily report . I couldn't find any cloud data fusion libraries . Any suggestions please .

Bazzz420
  • 21
  • 4
  • Could you elaborate a bit? You mention you couldn't find any libraries, you want to achieve it using some API? Do you want to check all jobs using CLI or Monitoring Console? You want to use piepline studio? Any specific reason why you want to have list of all pipelines? – PjoterS May 24 '21 at 13:00
  • @PjoterS I want to check all the jobs using CLI – Bazzz420 May 24 '21 at 14:37
  • Not sure if that functionality exist. Did you consider to use GCP UI to export pipelines in JSON format or Retrieve all pipelines from Cloud Data Fusion? – PjoterS May 25 '21 at 13:46
  • Similar thread and working solution. https://stackoverflow.com/questions/58942876/access-cdap-rest-api-of-a-cloud-data-fusion-instance – Bazzz420 May 25 '21 at 17:16

1 Answers1

1

Posting Community Wiki for better visibility.

As OP mentioned in the comment section, there was a similar thread which also can be a solution for this question. It can be found here - Access CDAP Rest API of a Cloud Data Fusion Instance.

Additional information which might be useful for getting all pipelines, batch pipelines or real-time pipelines can be found in GCP docs - Retrieve pipelines. Another possibility would be to Export pipelines from UI using JSON format.

PjoterS
  • 12,841
  • 1
  • 22
  • 54