0

I'm trying to configure livy with Zeppelin following this docs: https://zeppelin.apache.org/docs/0.7.3/interpreter/livy.html

However when I run:

%livy.spark
sc.version

I got the following error:

java.lang.RuntimeException: No result can be extracted from 'java.lang.NoSuchMethodException: org.apache.spark.ui.SparkUI.appUIAddress()', something must be wrong

I use Zeppelin 0.7.3, Spark 2.2.1, and Livy 0.4.0. Spark running on YARN (hadoop 2.9.0). This is vanilla install, I don't use distribution like cloudera/HDP. All these software runs on one server.

I can run example org.apache.spark.examples.SparkPi in spark-shell with --master yarn without any problem. So I confirm that spark is running well on YARN.

Any help would be appreciated.

Thanks, yusata.

MichaelChirico
  • 33,841
  • 14
  • 113
  • 198
Yusata
  • 199
  • 1
  • 3
  • 16
  • _"Spark running on YARN (hadoop 2.9.0)."_ Are you sure about that?! That's not possible (and am not talking about Hadoop 2.9.0 itself which is also unsupported). – Jacek Laskowski Jan 03 '18 at 11:47
  • Yes I'm pretty sure. hdfs version & spark sc.version show 2.9.0 and 2.2.1 respectively. Besides, I can run simple calculation from spark-shell (--master yarn) using csv file in HDFS. – Yusata Jan 04 '18 at 02:37

1 Answers1

0

This problem results from a method depreciation in spark 2.2.The appUiAddress no longer exist in spark 2.2.

As you can see in this PR https://github.com/apache/zeppelin/pull/2231. This issue is already solved.

Somehow you still encounter the problem. I think either downgrade Spark or use a newer version of Zeppelin could solve the problem.

Heyang Wang
  • 360
  • 2
  • 4
  • 19