Stack Overflow
Stack Overflow

Questions tagged [spark-thriftserver]

95 questions
0
votes
1 answer

How to monitor hive thrift server health in spark hosted in AWS EMR cluster?

We use a JDBC to connect to the hive thrift server on our spark EMR cluster. We have had some transient issues of the thrift server crashing and requiring manual restart. We are wondering if there is any [hopefully simple] way for us to monitor the…
James Wierzba
  • 16,176
  • 14
  • 79
  • 120
0
votes
0 answers

Writing to SQL Database - ConnectionException - PySpark

I'm attempting to write a dataframe to OmniSci's database, basically a GPU-backed SQL database. I'm receiving a strange error when attempting to do so: subs_dim.write.format("jdbc").option("url",…
DataScienceAmateur
  • 178
  • 2
  • 4
  • 12
0
votes
2 answers

Spark Thrift server load full dataset into memory before transmission via JDBC

Spark Thrift server trying to load full dataset into memory before transmission via JDBC, on JDBC client I'm receiving error: SQL Error: org.apache.spark.SparkException: Job aborted due to stage failure: Total size of serialized results of 48 tasks…
Triffids
  • 155
  • 1
  • 1
  • 16
0
votes
0 answers

How do I use SparkSQL and its execution engine to query Hive databases and tables without invoking any part of the Hive execution engine?

I've created select and join statements that I can run from the Hive CLI and/or the beeline CLI and/or Spark (2.3.1) WITH enableHiveSupport=TRUE. (Note: I'm using SparkR for my API) The join and write using beeline takes 30 minutes, but the join and…
quickreaction
  • 675
  • 5
  • 17
0
votes
1 answer

Achieving concurrency through FAIR scheduling in Spark

My Environment: I'm trying to connect Cassandra through Spark Thrift server. Then I create a Meta-Table in Hive Metastore which holds the Cassandra table data. In a web application I connect to Meta-table through JDBC driver. I have enabled fair…
naveendataer
  • 21
  • 3
0
votes
0 answers

HiveThriftServer2 only compatible with hive 1.2.1 jars

I am starting the HiveThriftServer in Spark application by using : HiveThriftServer2 .startWithContext(session.sqlContext()); I can see that i need to include hive-jdbc-1.2.1.spark2, hive-exec-1.2.1.spark2, hive-metastore-1.2.1.spark2 jars in the…
Amanpreet Khurana
  • 549
  • 1
  • 5
  • 17
0
votes
0 answers

Issue in using beeline to connect to Spark Thrift Server

I have started the spark thrift server on port 10015. It started successfully and also I can connect to metastore,but I'm unable to use beeline to connect to Spark thrift server. Please tell me if I have missed anything. -------Process I followed is…
hu li
  • 11
  • 2
0
votes
2 answers

JMeter 3.3 connect Spark 2.2.1 error: "Cannot create PoolableConnectionFactory (Method not supported)"

Use this list of jars, I can successfully connect SQuirrel SQL to Spark…
robert
  • 397
  • 1
  • 3
  • 14
0
votes
1 answer

Spark Thrift Server and ODBC

I have Spark 2.2 installed but not Hive and I would like to expose Spark tables through ODBC. I am able to start thrift server , with apparently no errors and my ODBC driver application is able to connect to thrift sever, but can’t see any Spark…
Paulo
  • 1
  • 3
0
votes
1 answer

Apache Spark thrift server error: Kerberos principal should have 3 parts

I am trying to start the Spark thrift server with Kerberos authentication enabled. To do so, I have these configurations in conf/hive-site.xml: