2

I have a kerberized cluster and want to run Spark programs as the "OS user" using Livy. Using the proxyUser option only sets the YARN user to the proxy User, the OS user is still Livy.

If this is not possible then can someone point me to the Livy code where the "spark-submit" takes place to create the new YARN application. I can check as to how this could be modified.

Tech Guy
  • 21
  • 2

1 Answers1

0

So Livy by default runs in client mode, and in this mode the driver runs as Livy. I could make it work as the impersonated user by granting sudo privileges to Livy, and then running as the impersonated user.

The easier way is to let YARN do it in cluster mode. This way Livy runs the driver as the impersonated user.

Tech Guy
  • 21
  • 2