1

I'm trying to create a pipeline in Vertex AI with kfp using my own components from local notebook in Spyder.

When I run the following piece of code:

@component(base_image="python:3.9", packages_to_install=["pandas"])
def create_dataset(
    gcs_csv_path_train: str,
    dataset: Output[Dataset],
):
    import pandas as pd
    df = pd.read_csv(gcs_csv_path_train)
    dataset = df.pop('Class')

I get the following error:

OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '<ipython-input-11-b28c15ec667f>'

The error is not raised if I use a Jupyter notebook online.

What am I doing wrong? Thanks.

Marco Abbatangelo
  • 77
  • 1
  • 11
  • What is the value of **gcs_csv_path_train**? Does it follow the Windows file naming scheme? – John Hanley Nov 04 '21 at 19:03
  • @JohnHanley gsc_csv_path_train is the path to the file in the google cloud storage bucket and it is like that: gsc_csv_path_train = f"bucket/folder/file.csv". In the Traceback the error points to the line: def create_dataset( – Marco Abbatangelo Nov 05 '21 at 08:37
  • Paths for Google Cloud storage look like this: **gs://bucket/folder/file.csv** The error message means that the filename is being intercepted as a file on a Windows filesystem. – John Hanley Nov 05 '21 at 08:54

1 Answers1

0

You need to check the file path that you use in your code. Because there are some characters that are not accepted such as the colon “ : ” in Windows file names. You can see more documentation about Windows standar paths.

When using the path in the python code, please follow below:

  • Use ‘r’ before any path- The r is a string literal that lets treat any string as a raw string, which means all escape codes will be ignored.
  • Use either double quotes “file-path” or single quote ‘file-path’ to specify the path.
  • Don’t use a combination of both “” or ‘

Correct file path is as below

filepath = r'C:\Test\file\file-input-thecodebuzz.txt'

OR

filepath = r"C:\Test\file\file-input-thecodebuzz.txt"

You can see more documentation.

Raul Saucedo
  • 1,614
  • 1
  • 4
  • 13