Spark Thrift Server - Unable to start (Hive error)

Question

I'm trying to launch the Spark Thrift server (Spark 1.2) on HDP 2.2.0 with Hive 0.14 (latest HortonWorks distribution) but unfortunately I'm getting this error that is quite cryptic for me :

Exception in thread "main" java.lang.RuntimeException: java.lang.NumberFormatException: For input string: "5s"
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)
        at org.apache.spark.sql.hive.HiveContext$$anonfun$4.apply(HiveContext.scala:235)
        at org.apache.spark.sql.hive.HiveContext$$anonfun$4.apply(HiveContext.scala:231)
        at scala.Option.orElse(Option.scala:257)
        at org.apache.spark.sql.hive.HiveContext.x$3$lzycompute(HiveContext.scala:231)
        at org.apache.spark.sql.hive.HiveContext.x$3(HiveContext.scala:229)
        at org.apache.spark.sql.hive.HiveContext.hiveconf$lzycompute(HiveContext.scala:229)
        at org.apache.spark.sql.hive.HiveContext.hiveconf(HiveContext.scala:229)
        at org.apache.spark.sql.hive.HiveContext.runHive(HiveContext.scala:292)
        at org.apache.spark.sql.hive.HiveContext.runSqlHive(HiveContext.scala:276)
        at org.apache.spark.sql.hive.HiveContext.setConf(HiveContext.scala:248)
        at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:91)
        at org.apache.spark.sql.SQLContext$$anonfun$2.apply(SQLContext.scala:90)
        at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
        at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
        at org.apache.spark.sql.SQLContext.<init>(SQLContext.scala:90)
        at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:72)
        at org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:51)
        at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:56)
        at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:358)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.NumberFormatException: For input string: "5s"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Integer.parseInt(Integer.java:492)
        at java.lang.Integer.parseInt(Integer.java:527)
        at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1094)
        at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1211)
        at org.apache.hadoop.hive.conf.HiveConf.getIntVar(HiveConf.java:1220)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:58)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)
        ... 26 more

Does anyone can help me fixing this issue? Same happens when I try to execute any command from the Spark shell related to Hive (like creating table, inserting data into existing one etc ..)

score 4 · Accepted Answer · edited Feb 05 '16 at 19:18

4

You should review your configuration.

Check if you can issue commands from Hive.
If no, look for string "5s" in hive-site.xml

edited Feb 05 '16 at 19:18

Mogsdad

44,709
21
151
275

answered Feb 02 '15 at 10:22

Harman

751
1
9
31

Thank you a lot mate! I have been struggling with this error for a couple of days and I really couldn't figure out what was happening. Damn config files! You made my day :) – Vittorio Cozzolino Feb 02 '15 at 10:40
@Vittorio No worries bro :) I'm glad this helped you out – Harman Feb 02 '15 at 10:48

score 3 · Answer 2 · answered Apr 20 '15 at 09:37

I have faced the same BUG. HDP2.2.4 have a wrong config file for hive

<property>
  <name>hive.metastore.client.connect.retry.delay</name>
  <value>5s</value>
</property>

should be

<value>5<values>

Hontonworks forum is very slow for user from china. So I can't post this bug to them =_="

Spark Thrift Server - Unable to start (Hive error)

2 Answers2