Stack Overflow
Stack Overflow

Questions tagged [spark-thriftserver]

95 questions
3
votes
2 answers

Caching DataFrame in Spark Thrift Server

I have a Spark Thrift Server. I connect to the Thrift Server and get data of Hive table. If I query the same table again, it will again load the file in memory and execute the query. Is there any way I can cache the table data using Spark Thrift…
Aditya Calangutkar
  • 486
  • 1
  • 6
  • 21
3
votes
1 answer

Giving spark thrift server the path to warehouse directory

I have setup my spark cluster and I am successful in connecting Tableau through Spark SQL connector. I created my tables from spark shell and saved dataframes from MySQL using (saveAsTable). How can I access the tables that I have saved from…
jdk2588
  • 782
  • 1
  • 9
  • 23
2
votes
2 answers

Getting error when querying iceberg table via Spark thrift server using beeline client?

I am trying to query iceberg table (External table with data in S3 & Metadata in Hivemetastore) using spark thrift server coming as part of Spark. I am able to query non iceberg tables but when I query iceberg table I am getting below error. Can we…
Bill Goldberg
  • 1,699
  • 5
  • 26
  • 50
2
votes
0 answers

Getting column names in the rows when querying Thrift database from pyspark

I have a Thrift database running on Apache Spark 3.1.2 where I have created a Table and inserted values using beeline. It looks like this: 0: jdbc:hive2://localhost:10000/> select * from mydb4.test; +-------+--------+ | key | value …
toerq
  • 117
  • 2
  • 10
2
votes
1 answer

incremental model in dbt with struct fields

I'm evaluating DBT for a possible usecase, and everything seems just fine except for one single situation. This is when the source table has struct fields. I'm using Spark Thrift Server connector, underlying data is stored as parquet in S3. DBT…
SCouto
  • 7,808
  • 5
  • 32
  • 49
2
votes
0 answers

NullPointerException in Spark Thrift Server using Apache Superset or Redash

I get a NullPointerException after connecting BI tools like Redash or Superset to a Spark Thriftserver (both tools use PyHive). Apache Zeppelin works fine for queries using STS and I could never reproduce the error there (Zeppelin uses…
Daniel Müller
  • 426
  • 1
  • 5
  • 19
2
votes
1 answer

Error executing MSCK REPAIR TABLE on external Hive table (Hive 2.3.6)

I have created an external Hive table based on a partitioned Parquet file in S3 using a statement similar to this: CREATE EXTERNAL TABLE default.person ( first_name STRING, last_name STRING ) PARTITIONED BY (age INT) STORED AS…
Phil
  • 598
  • 1
  • 9
  • 21
2
votes
0 answers

Very slow connection between Power Bi and Spark Thrift Server Using Spark Connector

I am using the "spark" connector in Power Bi desktop application to connect to spark thrift server using "HTTP" connection mode with "SSL" enabled and "Direct Query" option. The connection went successfully however, there is a too long time latency…
DigitalFox
  • 1,486
  • 1
  • 13
  • 17
2
votes
1 answer

Start Spark Thrift Server in Windows 10

I'm trying to start Spark Thrift Server using D:\spark\spark-2.3.2-bin-hadoop2.7\bin>spark-class org.apache.spark.deploy.SparkSubmit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 spark-internal in cmd. However, after I reach the…
user3344443
  • 475
  • 2
  • 11
  • 29
2
votes
1 answer

Authentication and authorization Spark Thrift Server

I am using HDP 2.5.0. I try to implement Spark Thrift Server authorization using tutorial: Apache Spark Fine Grain Security with LLAP Test Drive It working correctly, when I using username to login (not have password) and execute HiveQL. But I…
Piu
  • 135
  • 2
  • 20
2
votes
0 answers

Can't create table on apache spark ThriftServer through Spark Java API

I have a simple Spark API java program described as follows: SparkSession spark = SparkSession .builder() .appName("spark") .master("local") .getOrCreate(); ArrayList tagClass = new…
Taua Negri
  • 71
  • 5
2
votes
1 answer

ODBC configuration to connect to Spark Thrift Server

This question might seem repeated, in fact, I've seen a couple of questions related to this but not exactly with the same error, so I'm asking to see if anyone has a clue. I've set up a Spark Thrift Server running with default settings. Spark…
Daniel Martínez
  • 135
  • 2
  • 9
2
votes
2 answers

how to configure spark sql thrift server

I am trying to install apache thrift server . I installed spark,hadoop and yarn. I configured hive metastore and successfully started both metastore and hiveserver2: nohup hive --service metastore > /var/log/hive/metastore.log & nohup hive…
DarkSpark
  • 123
  • 3
  • 11
2
votes
4 answers

How to use Hive Metastore with MySQL (for Thrift Server or spark-shell)?

I'm using Apache Spark 2.1.1 and I'd like to set it up with an external Hive metastore (for Spark Thrift Server specifically). I have added hive-site.xml to $SPARK_HOME/conf folder that is as follows: …
José Carlos Guevara Turruelles
  • 481
  • 8
  • 23
2
votes
0 answers

Spark 2.0 Thrift server not started in yarn mode

I have started the spark-2.0 thrift server in local environment and it's working fine, when I'll tried with the cluster environment the following exception thrown. 16/06/02 10:21:06 INFO spark.SparkContext: Successfully stopped…
Churchill vins
  • 408
  • 1
  • 4
  • 12
1
2
3 4 5 6 7