Stack Overflow
Stack Overflow

Questions tagged [spark-thriftserver]

95 questions
1
vote
1 answer

Cassandra and Spark Thrift Server Integration

I'm trying to integrate Cassandra and Spark Thrift server. I followed the steps from here I get the following error while registering the cassandara tables in beeline console. Error: Error while compiling statement: FAILED: ParseException line 1:23…
Naveen
  • 11
  • 1
1
vote
1 answer

Connection to spark and accessing hive table without thrift server

I am writing a Java Spark application that needs to connect to hive and get some basic table info and query that table for data. I am creating a spark session and getting info like below. But this uses thrift server. I want to see if I can do the…
pkscoder
  • 41
  • 3
1
vote
1 answer

Hive - insert into table partition throwing error

I am trying to create a partitioned table in Hive on spark and load it with data available in other table in Hive. I am getting following error while loading the data: Error: org.apache.spark.sql.AnalysisException: …
kapil
  • 41
  • 1
  • 4
1
vote
0 answers

How can I set the fetch size when getting results from Spark Thrift Server using JDBC?

I have tried using statement.setFetchSize(required number), but that works when I connect to Hive using JDBC and not when I try going through the spark thrift server. My query produces a large result set, causing a OOM on the thrift server. Is there…
user9024779
  • 87
  • 8
1
vote
1 answer

Spark Thrift maintain caches among sessions

In Spark Thrift, when using the beeline client, is it possible to: User connects to the server using beeline and creates a cached table. User connects again to the server using beeline (different session), and can use the cached table created in…
Luis Alves
  • 1,286
  • 12
  • 32
1
vote
1 answer

Spark Thriftserver stops or freezes due to tableau queries

The spark cluster (spark 2.2) is used by around 30 people via spark-shell and tableau (10.4). Once a day the thriftserver gets killed or freezes because the jvm has to many garbage to collect. These are the error messages that I can find in the…
cronoik
  • 15,434
  • 3
  • 40
  • 78
1
vote
0 answers

How can I use cookie with http Thrift?

So, I'm communicating with the server using THttpClient of Thrift. The server is already sending the set-cookie header, but I was not sure on how to extract that from the response (in client side), and send back that cookie as the header of the…
newbie
  • 65
  • 5
1
vote
1 answer

Not able to connect to thrift server from spark shell

I am trying to connect to spark thrift server via spark shell by using following command: val df = spark .read .option("url", "jdbc:hive2://localhost:10000") .option("dbtable", "people") .format("jdbc") .load error: not found:…
snehil pandey
  • 11
  • 5
1
vote
0 answers

Is there a way to partially cache a partitioned hive table in spark memory

I have data stored in orc in s3 partitioned like so: mydata/year=2017/month=8/day=1 I've created an external table in spark on top of this table: spark.catalog().createExternalTable(tableName, s3Path, "orc"); This allows me to query data in that…
TheTurkish
  • 176
  • 2
  • 8
1
vote
1 answer

Registering temp tables in ThriftServer

I am new to Spark and am trying to understand how (if at all) is it possible to register dataframes as temp tables in the Spark thrift server. To clarify, this is what I am trying to do: Submit an application that generates a dataframe and…
Dan Markhasin
  • 752
  • 2
  • 8
  • 20
1
vote
1 answer

Spark thrift server unable to start

I am running spark 1.5.2 thrift server with Hive-1.2.1 on secured yarn-2.7.2 in windows using below command spark-submit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --master yarn-client…
Kumar
  • 3,782
  • 4
  • 39
  • 87
1
vote
2 answers

Spark Thrift Server - Unable to start (Hive error)

I'm trying to launch the Spark Thrift server (Spark 1.2) on HDP 2.2.0 with Hive 0.14 (latest HortonWorks distribution) but unfortunately I'm getting this error that is quite cryptic for me : Exception in thread "main" java.lang.RuntimeException:…
Vittorio Cozzolino
  • 931
  • 1
  • 14
  • 31
0
votes
0 answers

Spark SQL adding double quotes on column names when saving dataframe to table with JDBC

I'm playing with Spark Thrift Server and want to save a dataframe to it via the JDBC port it opens, but i'm facing an error with the SQL code generated by Spark. I've started the Thrift Server with the following command: spark-submit --master…
Gabriel Sandoli
  • 23
  • 3
0
votes
0 answers

Spark Thrift Server fails to start

I am trying to use logback with spark-thrift-server to log the application logs in elasticsearch and for that i am using log4j-over-slf4j along with logback-classic and logback-core. Service fails to start with this Exception:- Exception in thread…
Surya
  • 3
  • 4
0
votes
0 answers

ldap filtering not working for spark thrift application

I am using spark on Kubernetes with Thrift as a spark submit application. https://spark.apache.org/docs/latest/running-on-kubernetes.html I am able to enable LDAP authentication which allows the user to connect to thrift using beeline, and AD…
Conundrum
  • 43
  • 1
  • 6
1 2 3
4
5 6 7