4

In the following code I'm trying to start a hive thrift server from spark:

val conf = new SparkConf().setAppName("HiveDemo")

val sc = new SparkContext(conf)
val sql = new HiveContext(sc)

sql.setConf("hive.server2.thrift.port", "10001")

val df = sql.read.parquet("s3n://...")
df.registerTempTable("test")

HiveThriftServer2.startWithContext(sql)

while (true) {
  Thread.`yield`()
}

And I'm running this code as a step on an EMR cluster with the following configs:

emr-4.6.0
spark 1.6.1
hive 1.0.0

And I'm attaching to the cluster to test if the table is created with beeline using:

!connect jdbc:hive2://localhost:10001

I get Error: Could not open client transport with JDBC Uri: jdbc:hive2://localhost:10001: java.net.ConnectException: Connection refused

Is there an additional setup step that I'm missing on EMR?

djWann
  • 2,017
  • 4
  • 31
  • 36

1 Answers1

2

Actually this problem is fixed in Hive 1.3.0 and 2.0.0. So upgrade your version.

This problem occurs because Beeline's use of failed connection(s) causes failures and leaks.. All(error reproducing, how it is solved and which versions are containing the fix) are included in this link.

Hope it will help you.

UPDATE1:

is your hive Server is running in HTTP mode?

Connection URL When HiveServer2 Is Running in HTTP Mode:-

jdbc:hive2://<host>:<port>/<db>;transportMode=http;httpPath=<http_endpoint>

where:-

<http_endpoint> is the corresponding HTTP endpoint configured in hive-site.xml. Default value is cliservice.

Default port for HTTP transport mode is 10001.

Resource Link:

  1. Hive Client Details
  2. Hive JDBC : Could not open client transport with JDBC Uri
SkyWalker
  • 28,384
  • 14
  • 74
  • 132