How can I pass the list argument to vertex AI with gcloud cli when creating custom training?
I am using gcloud ai custom-jobs create command to create custom training job.
When using gcloud CLI of AI platform, it was simply like the below and didn't need to escape anything. However, in vertex AI the following error occurs when passing the list argument.
gcloud CLI (AI platform)
gcloud ai-platform jobs submit training {JOB_ID}}
...
--info-columns=['column_01','column_02']
...
gcloud CLI (Vertex AI)
gcloud ai custom-jobs create \
--region us-central1 \
--display-name=[DISPLAY_NAME] \
--project=[PROJECT] \
--python-package-uris='[BUCKET_URI]/trainer-0.1.tar.gz' \
--worker-pool-spec=machine-type=e2-standard-4,replica-count=1,executor-image-uri='us-docker.pkg.dev/vertex-ai/training/tf-cpu.2-11:latest',python-module=trainer.task \
--args=--sample-int=100 \
--args=--sample-string=12 \
--args=--sample-bool=True \
--args=--sample-columns=['column_01','column_02','column_03']
# --args=--sample-columns="['column_01','column_02','column_03']" # tried
# --args=--sample-columns=['column_01'\,'column_02'\,'column_03'] # tried
Error:
inside trainer/task.py
import argparse, ast
def parse_arguments():
def str2bool(v):
if isinstance(v, bool):
return v
elif v.lower() in ('true', 't'):
return True
elif v.lower() in ('false'):
return False
def str2list(s):
v = ast.literal_eval(s)
return v
parser = argparse.ArgumentParser()
parser.add_argument('--sample-int', type=int, default=5)
parser.add_argument('--sample-string', type=str, required=False)
parser.add_argument('--sample-bool', type=str2bool, required=False)
parser.add_argument('--sample-columns', type=str2list, default=['column_01', 'column_02'])
args = parser.parse_args()
return args
I am expecting something like escaping command like this, but couldn't find one for Vertex AI. I have checked [post_01], [post_02], but I couldn't find a similar case.
