sql query to spark-cassandra cluster thriftserver does not return

Question

I have a spark-cassandra cluster set up. cqlsh works just fine. I started the thriftserver and use beeline command line to connect to it. I can do

use schemaname
show schemas
show tables

but a simple

select * from tablename

gets stuck.

I tried a java client going thru the jdbc:hive2 interface, it shows the same behavior. Good results on show tables, but no return on a simple select statement.

The spark console shows all the jobs with no sign of complaint. the job is stuck on the Compiled stage.

Any one knows how to diagnose this deeper?

score 0 · Answer 1 · edited May 03 '16 at 09:13

0

create temporary table in sql for cassandra table . like following way

scala> sqlContext.sql(
   """CREATE TEMPORARY TABLE words
     |USING org.apache.spark.sql.cassandra
     |OPTIONS (
     |  table "words",
     |  keyspace "test",
     |  cluster "Test Cluster",
     |  pushdown "true"
     |)""".stripMargin)
scala> val df = sqlContext.sql("SELECT * FROM words")
scala> df.show()

edited May 03 '16 at 09:13

JensG

13,148
4
45
55

answered May 03 '16 at 06:29

mahendra singh

384
1
13

score 0 · Accepted Answer · answered May 04 '16 at 00:18

I am fine with the coding. This problem is solved. It is because this set up does not go thru yarn, and the first instance of thriftserver hogged up all cores in the cluster. I fired the sql against the second thriftserver instance which has 0 cores at its disposal.

sql query to spark-cassandra cluster thriftserver does not return

2 Answers2