Spark thrift server unable to start

Question

I am running spark 1.5.2 thrift server with Hive-1.2.1 on secured yarn-2.7.2 in windows using below command

spark-submit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --master yarn-client "C:\Spark\lib\spark-hive-thriftserver_2.10-1.5.2.jar"

It stopped with below exception,

16/04/11 12:31:00 INFO AbstractService: Service:HiveServer2 is started.
16/04/11 12:31:00 INFO HiveThriftServer2: HiveThriftServer2 started
16/04/11 12:31:00 ERROR ThriftCLIService: Error starting HiveServer2: could not start ThriftBinaryCLIService
org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address hostname1/192.168.65.7:10000.
    at org.apache.thrift.transport.TServerSocket.<init>(TServerSocket.java:109)
    at org.apache.thrift.transport.TServerSocket.<init>(TServerSocket.java:91)
    at org.apache.thrift.transport.TServerSocket.<init>(TServerSocket.java:87)
    at org.apache.hive.service.auth.HiveAuthFactory.getServerSocket(HiveAuthFactory.java:241)
    at org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:66)
    at java.lang.Thread.run(Thread.java:744)
16/04/11 12:31:00 INFO HiveServer2: Shutting down HiveServer2
16/04/11 12:31:00 INFO AbstractService: Service:ThriftBinaryCLIService is stopped.

How to solve this.

Thanks.

Hive server2 running successfully in my machine, but spark again trying to start hive server2. — Kumar, Apr 11 '16 at 09:57
So that's it :), Hiveserver is using the port 10000, you should change the port of Spark Thrift server to somoething like 10001 for example — user1314742, Apr 11 '16 at 09:58

user1314742 · Accepted Answer · 2016-04-11T10:12:38.843

Possible cause of the problem is that the port 10000 is already in use (as mentioned in your comment that Hiveserver is already running, which uses by default the port 10000).
You could change it (to 10005 for example) when running thrift server.

I would recommend that you start the thrift server as follow:

$cd $SPARK_HOME
$./sbin/start-thriftserver.sh --hiveconf hive.server2.thrift.port=10005 --master  yarn-client

Please refer to the documentation here

Spark thrift server unable to start

1 Answers1