1

Im trying to execute the following curl command to run a job:

curl -k --basic --user 'user:psw' -d 'input.string= {"user":13}' 'https://localhost:8090/jobs?appName=test&classPath=test.ImportCSVFiles&context=import&sync=true'

But I get the following error:

 "com.typesafe.config.ConfigException$WrongType: String: 1: input.string has type OBJECT rather than STRING"

My idea is to pass more than one parameter like an sql query. A json format to easy handling on my submitted jar.

I'm on the right way or there is another way?

José Carlos Guevara Turruelles
  • 481
  • 8
  • 23
  • I am not sure what you want to achieve, but exception is pretty clear. {"user":13} is a json object, and "input.string" wants to except a string value. So send it as string escape quotes if necessary, and internally convert that string to json – Abhishek Anand Dec 04 '16 at 17:37
  • @AbhishekAnand that is what I have also tried but if I use `"input.string = {\"user\":13}"` I get the same result and using `"input.string = \{\"user\":13\}"` then _Reserved character '\\' is not allowed outside quotes_. From my point of view _{"user":13}_ is a string, it have json format? Yes, but it is an string. – José Carlos Guevara Turruelles Dec 05 '16 at 08:59
  • did you try 'input.string= \"{\"user\":13}\" ' – Abhishek Anand Dec 05 '16 at 10:31
  • @AbhishekAnand Thanks that works!!! One more question, the **input.string** it is a reserved word? Can be used like **input.json** or something else can be done with it? – José Carlos Guevara Turruelles Dec 05 '16 at 11:49

2 Answers2

1

input.string is not a reserved keyword -- in fact, you can name the parameters arbitrarily. Let say, you POST two parameterss foo.string and foo.number. You then read the parameters in your SJS job, like this: 

 // run is the starting point of a SJS job (also see validate function)
 override def runJob(sc: SparkContext, config: Config): Any = {
  val cmd = config.getString("input.cmd") 
  val fooString = config.getString("foo.bar")
  val fooNum    = config.getInt("foo.number")

Just in case you plan to execute SJS jobs from Scala/Java:

Apache Commons Lang (org.apache.commons.lang3) comes with a very helpful class to escape JSON: StringEscapeUtils

I use it to escape inputs from my Scala application that I need to pass to SparkJobServer jobs, like this:

input.schema=\"" + StringEscapeUtils.escapeJson(referenceSchema)+"\""

referenceSchema is a JSON document (in my case a JSON array)

input.schema is then one of many comma-separated parameters in the body of HTTP post from Scala...

Tim Dörnemann
  • 11
  • 3
1

Copying it from comment.

Issue was json was trying to read it not as string but as object directly because of string starting with braces "{".

Correct input 'input.string= \"{\"user\":13}\" '

Abhishek Anand
  • 1,940
  • 14
  • 27