0

I use Spark 1.3.1.

How to store/save a DataFrame data to a Hive metastore?

In Hive If I run show tables the DataFrame does not appear as a table in Hive databases. I have copied hive-site.xml to $SPARK_HOME/conf, but it didn't help (and the dataframe does not appear in Hive metastore either).

I am following this document, using spark 1.4 version.

dataframe.registerTempTable("people")

How to analyze the spark table in Hive?

Jacek Laskowski
  • 72,696
  • 27
  • 242
  • 420
Venu A Positive
  • 2,992
  • 2
  • 28
  • 31
  • Are you sure your hive is configured properly? Are you able to query hive tables from beeline or hive shell? – karthik manchala Nov 18 '15 at 10:41
  • yes, if i run sqlContext.sql("CREATE TABLE IF NOT EXISTS weather(dates STRING, temp STRING, wind STRING, atmosPressure STRING)") It's creating the table. So It's not a problem. – Venu A Positive Nov 18 '15 at 12:03

2 Answers2

0

You can use insertInto method to save dataframe in hive.

Example:

dataframe.insertInto("table_name", false);
//2nd arg is to specify if table should be overridden
karthik manchala
  • 13,492
  • 1
  • 31
  • 55
0

Got the solution. I am using spark 1.3.1, so that it's not supporting all. Now using spark 1.5.1 that problem resolved. I have noticed data-frames fully working after 1.40. Many commands are deprecated.

Venu A Positive
  • 2,992
  • 2
  • 28
  • 31