0

i have made a simple python file which want to submit in Livy.Livy is currently running in local mode.Also I have mentioned following property in livy.conf file.

Property name: livy.file.local-dir-whitelist, 
value "/usr/local/livy/scripts"

My file is kept in following path "/usr/local/livy/scripts"

import json, pprint, requests, textwrap
host = 'http://localhost:8998'
data = {'kind': 'spark'}
headers = {'Content-Type': 'application/json'}
r = requests.post(host + '/sessions', data=json.dumps(data), headers=headers)
r.json()

I am submitting it using curl as follows:

curl -X POST --data '{"file": "/usr/local/livy/scripts/pi.py"}' -H "Content-Type: application/json" 10.140.178.24:8999/batches

It is giving me following error:

requirement failed: Local path /usr/local/livy/scripts/pi.py cannot be added to user sessions.

My Ubuntu system only have following things:

a)Spark
b)Livy
c)Java

What am I doing wrong here?

MichaelS
  • 5,941
  • 6
  • 31
  • 46
Utkarsh Saraf
  • 475
  • 8
  • 31

2 Answers2

0

For people using incubating mode of livy for first time,kindly check that the template file is renamed with stripping off .template in livy.conf.template.Then make sure that the following configurations are present in it.

livy.spark.master = local
livy.file.local-dir-whitelist = /path/to/script/folder/

Kindly make sure that forward slash is present in end of path

Then write url in following manner for

Python:

curl -v -X POST --data '{"file": "/path/to/script/folder/name-of-python-file.py"}' -H "Content-Type: application/json" localhost:8998/batches

Note:It will not accept relative path,whole absolute path needs to be defined in it.

Utkarsh Saraf
  • 475
  • 8
  • 31
-1

curl -X POST --data '{"file": "/usr/local/livy/scripts/pi.py"}' -H "Content-Type: application/json" 10.140.178.24:8999/batches {"id":2,"state":"starting","log":[]}

chitrendra chaudhary
  • 1