I would like to create a Synapse Pipeline for batch inferencing with data ingestion to store the data into data lake and then use this as input to call a batch endpoint already created (through ML Execute Pipeline) Then, capture the output into the data lake (appended to a table) to continue the next steps...
The documentation from Microsoft to setup such a scenario is very poor and everything I tried is failing.
Below is the Azure Machine Learning Execute Pipeline configuration. I need to pass the value for the dataset_param with data asset instance already available as below.
But, it complains that the dataset_param is not provided. Not sure, how to pass this value...
Here is the original experiment / pipeline / endpoint created by the DevOps pipeline. I just call this endpoint above from the Synapse Pipeline