Questions tagged [spark-thriftserver]
95 questions
1
vote
1 answer
Path based rules giving 404 for AKS ingress controller
I have two ClusterIP services on AKS and an ingress controller ingress-nginx-4.6.1 when I try the following config to route the traffic to each service I get 404.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: spark-ingress
…
Conundrum
- 43
- 1
- 6
1
vote
1 answer
Alter table command on complex data type column ( Nested struct ) not working in spark thrift server
I am trying to run an alter table command using beeline connecting to sparkt thrift server which is configured to use remote hive metastore. I am getting the following error.
Table is created using below command:
CREATE TABLE…
Bill Goldberg
- 1,699
- 5
- 26
- 50
1
vote
1 answer
Spark Thrift 3.2.2 impersonate user facing error with metastore authen. SASL negotiation failure, GSS initiate failed
On hadoop kerberized cluster. If im not impersonate user on spark thrift server. It work well. But when i do it. Im facing an error about authentication with metastore.
I flow this…
qxk71551
- 95
- 9
1
vote
0 answers
java.lang.ClassNotFoundException: org.apache.hive.service.cli.thrift.TCLIService$Iface
I am getting java.lang.ClassNotFoundException: org.apache.hive.service.cli.thrift.TCLIService$Iface exception in runtime when I call this API in the browser,…
Sun
- 3,444
- 7
- 53
- 83
1
vote
0 answers
Runtime Error: Cannot set database in spark! [DBT + Spark + Thrift]
Can anyone help me on this?
I'm getting error,***Runtime Error: Cannot set database in spark!*** while running dbt model via Spark thrift mode with remote Hive metastore.
I need to transform some models in DBT using Apache Spark as the adapter. Now,…
Jithin K J
- 11
- 3
1
vote
1 answer
spark ETL and spark thrift server
Some details:
Spark SQL (version 3.2.1)
Driver: Hive JDBC (version 2.3.9)
ThriftCLIService: Starting ThriftBinaryCLIService on port 10000 with 5...500 worker threads
BI tool is connect via odbc driver
After activating Spark Thrift Server I'm…
Alex k
- 25
- 5
1
vote
0 answers
advanced.session-leak after sometime of starting spark thrift server with datastax cassandra connector
Hi I am getting the following error after some time of inactivity.
Error: Error running query: com.typesafe.config.ConfigException$Missing: withValue(advanced.reconnection-policy.base-delay): No configuration setting found for key…
rk1234 afbce
- 33
- 5
1
vote
2 answers
trying to start spark thrift server with datastax cassandra connector
I have started spark-thrift server and connected to the thrift server using beeline. when trying to query create a table in hive metastore and i am getting the following error.
creating table
create table meeting_details using…
rk1234 afbce
- 33
- 5
1
vote
0 answers
Kubernetes Deployment fails with CrashLoopBackOff
Hi I am using the yaml file provided in this link
https://github.com/dirk1492/docker-spark/blob/master/kubernetes/thriftserver.yaml
and modified it to bring up Spark Thrift Server on my kubernetes cluster in the cloud.
However, getting this below…
Raghav Kapoor
- 13
- 2
- 5
1
vote
0 answers
SQL query issue on thrift server client
I have performed join on 2 tables memberstable (service_id,created_date) and member_servicestable (id and user_id) the join was made on registered id and id since they are the same. I want to get the total number of user ids for service id using a…
Srinidhi Nagabhushan
- 11
- 1
1
vote
1 answer
Spark Thrift server queuing up queries
When Parallel queries are hitting Spark Thrift server, in Spark UI --> JDBC/ODBC Server , it shows up all queries as started but all of them gets executed in a sequential manner
Here's the Thrift Server startup script---
start_thriftserver (){
…
Arun
- 31
- 2
1
vote
0 answers
How to use authentication on Spark Thrift server on Spark standalone cluster
I have a standalone spark cluster on Kubernetes and I want to use that to load some temp views in memory and expose them via JDBC using spark thrift server.
I already got it working with no security by submitting a spark job (pyspark in my case) and…
Bruno Faria
- 549
- 1
- 6
- 20
1
vote
3 answers
How to view pyspark temporary tables on Thrift server?
I'm trying to make a temporary table a create on pyspark available via Thrift. My final goal is to be able to access that from a database client like DBeaver using JDBC.
I'm testing first using beeline.
This is what i'm doing.
Started a cluster…
Bruno Faria
- 549
- 1
- 6
- 20
1
vote
1 answer
TBLPROPERTIES('skip.header.line.count'='1') is not working on sparkThrift connected from beeline with hive jdbc 1.2.1
I am using spark 2.3 and connecting sparkThrift with beeline.
Hive jdbc version 1.2.1
Spark SQL version 2.3.1
I am trying to create external table with skip header property but select command is always returning data with header as first row, below…
Mahesh J
- 94
- 1
- 9
1
vote
1 answer
How can I make my pyspark database visible to beeline on windows?
Hey All, I need some help to open a pickle jar in pyspark. I'm learning pyspark and as an exercise I'm trying to do something that I assumed would be a simple thing... but its giving me a very hard time.
So let's start with what I'm doing. I'm…
costargc
- 516
- 5
- 14