0

I have a Scala program wherein I create a hivecontext and run a hive query from it. The hive query is a multi-table insert query. I am facing issues where the query runs successfully but I cannot see the out being generated. Interesting thing is that if in the multi-table inert query I insert into one table the query is running successfully i.e I can see the out data. For example: 

val hiveContext = = new org.apache.spark.sql.hive.HiveContext(sc)

Query that is not generating data: 

hiveContext.sql("FROM source_table INSERT INTO destiantion_table_1 PARTITION (...)
INSERT INTO destination_table_2 PARTITION (...)
INSERT INTO destination_table_3 PARTITION (...)")

Query that is working fine: 

hiveContext.sql("FROM source_table INSERT INTO destiantion_table_1 PARTITION (...)

I looked into the executor logs there was no error reported for the query that did not execute.

I am running Spark 1.5 on YARN

Let me know if any additional information is required on the above .

greg-449
  • 109,219
  • 232
  • 102
  • 145
Sriram
  • 53
  • 7
  • So you assume that the Spark SQL compiler supports each and every fancy feature of the Hive SQL compiler (e.g. the multi-table insert using that bizarre "FROM - INSERT" syntax). That makes you an optimist. – Samson Scharfrichter Aug 09 '16 at 17:16
  • ok..please guide me on how i can determine which feature of hive is supported and which is not? thanks – Sriram Aug 10 '16 at 08:57
  • _"how i can determine ..."_ -- that's exactly what you are doing: trying something, checking whether it works as expected or not, trying sthg else. And start again with the next Spark release. – Samson Scharfrichter Aug 10 '16 at 11:04

0 Answers0