0

I installed cassandra spark-hadoop cluster on 3 ec2 nodes. Yesterday, I was able to start the spark thrift server on node0, and actually executed a simple sql statement in beeline. Today, after a schema change, I restarted the thrift server, now I get a SparkException java.lang.IllegalArgumentException: ip-172-30-4-140 at org.apache.hadoop.hive.cassandra.cql3.input.HiveCqlInputFormat.getRecordReader(HiveCqlInputFormat.java:212)

the ip-172-30-4-140 is simply the private ip of that node

I tried running the same sequence from the other two cassandra nodes, and for those, the sql statement gets stuck and never returns.

What is this error? any one knows?

Erick Ramirez
  • 13,964
  • 1
  • 18
  • 23
bhomass
  • 3,414
  • 8
  • 45
  • 75

2 Answers2

0

This is not ip . you have to put either private ip or public dns .

mahendra singh
  • 384
  • 1
  • 13
  • there is no input parameter for entering the ip. this message was generated internally without any input from me. – bhomass Apr 27 '16 at 20:49
  • any one knows where thriftserver got the private DNS from? this is what is returned when you do echo $HOSTNAME. I checked thru numerous config files. Can not figure out where thriftserver, spark, or hive is getting this from. – bhomass May 04 '16 at 00:21
0

ok, I found the problem.

The default value for the host parameter points to the internal ip DNS of the ec2, which causes the exception. It needs to be explicitly declared

sudo dse spark-sql-thriftserver start hive.server2.thrift.bind.host=your-ec2-private-ip

bhomass
  • 3,414
  • 8
  • 45
  • 75