3

I am connecting with Hive using Spark 2.x and I am running following Spark Query:

spark.sql("""DROP TABLE IF EXISTS db_name.table_name""")
spark.sql("""Create TABLE IF NOT EXISTS db_name.table_name""")

if the table doesn't exist then the first query gives exception of Table Does not exist. If Table exist and I am running the second query in the first place then it throws Table already exists exception.

Which Means the condition IF EXISTS and IF NOT EXISTS is not working.

I read it somewhere that there may be data nucleus dependency issue. so Below are the dependencies I am using for Data nucleus:

<dependency>
            <groupId>org.datanucleus</groupId>
            <artifactId>datanucleus-rdbms</artifactId>
            <version>3.2.9</version>
        </dependency>
        <dependency>
            <groupId>org.datanucleus</groupId>
            <artifactId>datanucleus-core</artifactId>
            <version>3.2.10</version>
        </dependency>
        <dependency>
            <groupId>org.datanucleus</groupId>
            <artifactId>datanucleus-api-jdo</artifactId>
            <version>3.2.6</version>
        </dependency>
Naman Agarwal
  • 614
  • 1
  • 8
  • 28
  • Define what DataNucleus has to do with this. You quote some SQL statements, and so are you executing them through DataNucleus for some reason? –  Sep 25 '18 at 12:21
  • I figured out a way of not using these conditions. But still these are not working – Naman Agarwal Oct 03 '18 at 07:09
  • And it apparently has nothing to do with DataNucleus, as per what the question asked. –  Oct 09 '18 at 08:36
  • Okay then what can be the possible cause for this? – Naman Agarwal Oct 09 '18 at 09:13
  • 1
    I simply asked you to clarify where "DataNucleus" is in this, and you didn't. I could have answered a specific question on DataNucleus. I dont use "Spark", or "Hive" or any of the other stuff here. –  Oct 09 '18 at 09:43
  • What is your RDBMS ? If you use mysql or sql server or something else this would influence the answer – CharybdeBE Aug 09 '21 at 11:30

0 Answers0