0

I am getting started with Apache Livy and I was able to follow online documentation and was able to submit the spark job through Curl(I have posted another question on converting curl to REST call). My plan was to tryout with curl and then convert that curl to a REST API call from Scala. However after spending entire day to figure out how to convert Livy curl call to REST, I feel like my understanding is wrong.

I am checking this example from Cloudera and I see we have to create a LivyClient instance and upload the application code to the Spark context from it. Is the correct way? I have a use case where I need to trigger my spark job from Cloud, do I need to put dependencies on Cloud and add it with uploadJar like it is mentioned in Cloudera example?

Explorer
  • 1,491
  • 4
  • 26
  • 67

1 Answers1

1

There are 2 options to interact with Livy server

  1. Using the Livy Client which makes it easier to interact with Livy Server.
  2. There are Rest API exposed which can be used programmatically.

Please check the below links

https://livy.incubator.apache.org/docs/latest/rest-api.html

dassum
  • 4,727
  • 2
  • 25
  • 38