3

From spark using: 

DataFrame.write().mode(SaveMode.Ignore).format("orc").saveAsTable("myTableName")

Table is getting saved I can see using below command's hadoop fs -ls /apps/hive/warehouse\test.db' where test is my database name

drwxr-xr-x - psudhir hdfs 0 2016-01-04 05:02 /apps/hive/warehouse/test.db/myTableName

but when I trying to check tables in Hive I cannot view them either with command SHOW TABLES from hiveContext.

sudhir
  • 1,387
  • 3
  • 25
  • 43
  • Is this a internal table?/or U have mentioned as external table some whee – Amaresh Jan 04 '16 at 14:36
  • I didn't get you, " myTableName " is the new table name , sorry I was using SaveMode.Ignore and SaveMode.Errorifxists as creating new table – sudhir Jan 04 '16 at 15:15
  • what is the content of your hive-conf file you have put under the Spark conf dir? Can you see from Spark Hive tables? – mgaido Apr 04 '16 at 07:11

3 Answers3

2
sudo cp /etc/hive/conf.dist/hive-site.xml /etc/spark/conf/

This worked for me in a Cloudera quick start Virtual Box.

mech
  • 2,775
  • 5
  • 30
  • 38
Tanaji Sutar
  • 109
  • 1
  • 8
1

You have to copy the hive-site.xml file (mine is located at /etc/hive/conf.dist/hive-site.xml) to Spark conf folder (mine is located at /etc/spark/conf/)

sudo cp /etc/hive/conf.dist/hive-site.xml /etc/spark/conf/

Restart Spark and it should work.

lmtx
  • 5,018
  • 3
  • 20
  • 29
-3

I think you need to run INVALIDATE METADATA; in the hive console to refresh the databases and view your new table.

Simon McGloin
  • 47
  • 4