SnappyData REST API to Submit Job

Question

I am trying to submit Snappy Job using REST API.

• We have been able to submit SnappyJob using snappy-job submit Command Line tool.
• I could not find any documentation how to do the same thing through REST API.

• I found somewhere mentioned in the forum that SnappyData is using the spark jobserver REST API.

  Could you point to the Documentation / User Guide how to do that?

Answer 1

Snappydata internally uses spark-jobserver for submitting the jobs. Hence, all the spark-jobserver REST APIs are accessible on Snappydata's lead node.

You can refer to all spark-jobserver API here: https://github.com/SnappyDataInc/spark-jobserver#api

Here are some useful curl commands to clarify it further:

• deploy application jar on job-server:

curl --data-binary @/path/to/applicaton.jar localhost:8090/jars/testApp

testApp is the name of the job server app which will be used to submit the job

• create context:

curl -X POST "localhost:8090/contexts/testSnappyContext?context-factory=org.apache.spark.sql.SnappySessionFactory"

testSnappyContext is the name of the context which will be used to submit the job.

Also, note that we are passing a custom context-factory argument here which is necessary for submitting snappy job.

• submit the job:

curl -d "configKey1=configValue1,configKey2=configValue2" "localhost:8090/jobs?appName=testApp&classPath=com.package.Main&context=testSnappyContext"

com.package.Main is the fully-qualified name of the class which is extending org.apache.spark.sql.SnappySQLJob.

• stop the job

curl -X DELETE localhost:8090/jobs/bfed84a1-0b06-47ca-81a7-9b8defb51e38

bfed84a1-0b06-47ca-81a7-9b8defb51e38 is the job-id which you will get in the response of job submit request

• stop the context

curl -X DELETE localhost:8090/contexts/testSnappyContext

• undeploying the application jar

The version of job-server being used by snappydata doesn't have a RESTful API exposed for undeploying the jar. However, deploying any jar with the same app name (testApp in our example) will override the previously deployed jar for the same app.