0

I am getting a value returned by hitting a HTTP endpoint which I am storing in a column. Now a want to trigger another Http Endpoint with the value in the column. But the HTTP endpoint takes hardcoded values or macros only. So I want to know if I can set a run-time argrument based on the column value

Please suggest How I can do this.

SUDHIR GARG
  • 75
  • 9

1 Answers1

0

There are few ways of setting runtime arguments:

  1. Using Argument Setter Plugin - type of Action plugin that allows one to create reusable pipelines by dynamically substituting the configurations that can be served by an HTTP Server.

  2. Specifying runtime arguments as a JSON map in the request body, when starting an program.

  3. CDAP Preferences HTTP RESTful API.

I recommend you check the official documentation for Datafusion, where you can find a way of setting variables/macros using the GCS bucket.

Additionally, have a look at the following thread on SO. It describes the method with specifying runtime arguments as a JSON map in the request body.

aga
  • 3,790
  • 3
  • 11
  • 18
  • Thanks for the answer, but with argument setter we have to make the object public I presume. So in google if a object is public and if you update it , it takes 1 hour to reflect the latest result in public URL. So how my pipeline takes the latest arguments in that case. – SUDHIR GARG Aug 20 '20 at 07:50