1

When Parallel queries are hitting Spark Thrift server, in Spark UI --> JDBC/ODBC Server , it shows up all queries as started but all of them gets executed in a sequential manner

Here's the Thrift Server startup script---

start_thriftserver (){
        sudo /usr/lib/spark/sbin/start-thriftserver.sh \
--master yarn \
--deploy-mode client \
--executor-memory 3200m \
--executor-cores 2 \
--driver-memory 4g \
--conf spark.dynamicAllocation.enabled=true \
--conf spark.shuffle.service.enabled=true \
--conf spark.serializer=org.apache.spark.serializer.KryoSerializer \
--conf spark.dynamicAllocation.schedulerBacklogTimeout=1s \
--conf spark.dynamicAllocation.minExecutors=50 \
--conf spark.executor.memoryOverhead=684
Arun
  • 31
  • 2

1 Answers1

0

This is indeed a confusing topic.

spark.sql.hive.thriftServer.singleSession=false

Try this.

That said, I am a little sceptical on all this.

thebluephantom
  • 16,458
  • 8
  • 40
  • 83