I am trying to create AWS Sagemaker Pipeline. ClientError: An error occurred (ValidationException) when calling the CreatePipeline operation:

Question

I am trying to create AWS Sagemaker Pipeline. I created this pipeline 2 months back and it was running then. But now I am getting following error on running

pipeline.upsert(role_arn=role)

ClientError: An error occurred (ValidationException) when calling the CreatePipeline operation: Unknown property reference [Parameters.DataSplitPercent].

Following is the some code written to create pipeline:

Input_data =ParameterString(name="InputDataUrl",default_value=f"s3://akgargbucket/iris.data")<br/>

data_split_percent = ParameterFloat(name="DataSplitPercent", default_value=0.7)<br/>

    framework_version="0.23-1",
        instance_type=processing_instance_type,
        instance_count=processing_instance_count,
        base_job_name=f"{base_job_prefix}/sklearn-abalone-preprocess",
        sagemaker_session=sagemaker_session,
        role=role)
    step_process = ProcessingStep(
        name="ReadAndEncodeData",
        processor=sklearn_processor,
        outputs=[
            ProcessingOutput(output_name="train", source="/opt/ml/processing/train"),
            ProcessingOutput(output_name="test", source="/opt/ml/processing/test"),
             ],
        code=os.path.join(BASE_DIR, "preprocess.py"),
        job_arguments=["--input-data", input_data,
        "--data_split_percent",data_split_percent])

score 3 · Answer 1 · answered Mar 08 '23 at 16:27

3

I encountered the same error. We need to add the parameter to the parameter list when creating the pipeline. For example:

from sagemaker.workfow.pipeline import Pipeline
pipeline = Pipeline(name=my_pipeline, 
                    parameters=[data_split_percent],
                    steps=my_step_list,
                    sagemaker_session=my_sagemaker_session)

answered Mar 08 '23 at 16:27

W. Dong

31
3

Sure do! Thanks! – matixsnow Jun 29 '23 at 21:18
Thanks for the reply, I had solved it with a different approach (mentioned earlier) but I will take a note of this as well. – akshat garg Jul 14 '23 at 11:37

score 0 · Answer 2 · answered Jun 14 '22 at 01:31

0

What version of the SageMaker Python SDK are you using?

import sagemaker
print(sagemaker.__version__)

I suggest testing by installing the latest version.

!pip install sagemaker --upgrade

answered Jun 14 '22 at 01:31

Marc Karp

949
4
6

akshat garg · Accepted Answer · 2022-07-15T12:01:53.217

I updated data_split_percent to data_split_percent = ParameterString(name="DataSplitPercent", default_value="0.7") and in the "preprocess.py" file "data_split_percent" is read as float, though it is mentioned as string in pipeline config file (as mentioned before "ParameterString"). Then, I restarted the kernel of my sagemaker notebook through which I was creating and running the pipeline. It started working fine.

I am trying to create AWS Sagemaker Pipeline. ClientError: An error occurred (ValidationException) when calling the CreatePipeline operation:

3 Answers3