3

I have setup my spark cluster and I am successful in connecting Tableau through Spark SQL connector.

I created my tables from spark shell and saved dataframes from MySQL using (saveAsTable).

How can I access the tables that I have saved from Tableau? Do I need to give a path of the warehouse directory when starting the spark thrift server? If yes, how it can be done and if no, how can this be done?

mrsrinivas
  • 34,112
  • 13
  • 125
  • 125
jdk2588
  • 782
  • 1
  • 9
  • 23

1 Answers1

2

Make sure you are pointing to the same metastore for spark-shell and thriftserver

Metastore sharing can be 2 ways, in simple

  1. Starting both shell and thrift from same location
  2. Setting up remote database for metastore

You can pass hive confs to Spark thrift server with --hiveconf and Spark confs with --conf

./sbin/start-thriftserver.sh \
  --conf spark.sql.warehouse.dir=path/to/warehouse/dir \
  --hiveconf hive.server2.thrift.port=<listening-port> \
  --hiveconf hive.server2.thrift.bind.host=<listening-host> \
  --master <master-uri>
  ...
mrsrinivas
  • 34,112
  • 13
  • 125
  • 125