1

I have created a job in DataBricks. Now, I'm trying to run it using databricks cli. See part of the job json:

"tasks": [
    {
        "task_key": "task_key",
        "python_wheel_task": {
            "package_name": "package_name",
            "entry_point": "entry_point",
            "parameters": [
                "command",
                "parameter1",
            ]
        },
        "libraries": [
            {
                "whl": "dbfs:/wheels/anme.whl"
        }
]

Command I'm using to run the job (this command works):

databricks jobs run-now --job-id 1111

Then, I try to change parameter1 by running

databricks jobs run-now --job-id 1111 --python-params '"["value1", "value2"]"'

But I get error:

Error: JSONDecodeError: Expecting value: line 1 column 2 (char 1)

how can I provide new parameters to python_wheel_task?

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
shalva_t
  • 83
  • 6

2 Answers2

1

On top the answer from Alex, I found that we have to use escape symbols for command to work.

the solution is:

databricks jobs run-now --job-id 1111 --python-params '[\"value1\", \"value2\"]'
shalva_t
  • 83
  • 6
  • Hmmmm, something is wrong then in CLI... let me ping developers, because API says that it's array of strings: https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsRunNow – Alex Ott Aug 12 '22 at 11:52
  • You can take a look at this article, where author describes similar issue I'm facing: https://endjin.com/blog/2019/07/azure-databricks-cli-error-jsondecodeerror-expecting-property-name-enclosed-in-double-quotes – shalva_t Aug 12 '22 at 13:35
  • @AlexOtt do you have any update from developers? – shalva_t Aug 24 '22 at 10:52
  • file github issue at databricks-cli repos – Alex Ott Aug 24 '22 at 14:01
0

The value for --python-params should be JSON array of strings. In your case you don't need the double quotes around array - just use '["value1", "value2"]'.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • Hey Alex, thanks for the answer. Can you please explain what you mean by JSON array of strings, and how should I format the input? Error: JSONDecodeError: Expecting value: line 1 column 2 (char 1) – shalva_t Aug 12 '22 at 11:07
  • Alex, I fixed the error with escape symbols. please see my answer as post – shalva_t Aug 12 '22 at 11:14