1

I am trying to export from hive table to teradata using TDCH connector, I am getting below errror:-

15/05/07 08:01:03 INFO tool.ConnectorExportTool: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException
            at java.lang.Class.forName0(Native Method)
            at java.lang.Class.forName(Class.java:190)
            at com.teradata.connector.common.tool.ConnectorJobRunner.runJob(ConnectorJobRunner.java:81)
            at com.teradata.connector.common.tool.ConnectorExportTool.run(ConnectorExportTool.java:61)
            at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
            at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
            at com.teradata.hadoop.tool.TeradataExportTool.main(TeradataExportTool.java:24)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
            at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.api.MetaException
            at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
            at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
            at java.security.AccessController.doPrivileged(Native Method)
            at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
            ... 13 more

I understand from error that hive-metastore jar is missing. But it is already there in hive/lib folder

 hive-metastore.jar -> hive-metastore-0.9.0.jar

Is already present in the path: /usr/hdp/2.2.4.2-2/hive/lib

Ronak Patel
  • 3,819
  • 1
  • 16
  • 29
priyanka
  • 305
  • 1
  • 3
  • 18

3 Answers3

1

The problem is that org/apache/hadoop/hive/metastore/api/MetaException class is not available to java runtime. can you ensure that the Jar is under valid runtime classpath.

few try outs can be try passing the classpath explicitly to jvm via -cp , alternatively you can try putting the jar in launch directory to ensure its present.

Akash Yadav
  • 2,411
  • 20
  • 32
1

You have to define environment variables LIB_JARS and HADOOP_CLASSPATH. You can use the environment variable LIB_JARS with the parameter -libjars

You can find an example under https://developer.teradata.com/sites/all/files/Teradata%20Connector%20for%20Hadoop%20Tutorial%20v1%200%20final.pdf.

According to the README, following jars are necessairy:

    Hive Job(version 0.11.0 as example):
         a) hive-metastore-0.11.0.jar
         b) hive-exec-0.11.0.jar
         c) hive-cli-0.11.0.jar
         d) libthrift-0.9.0.jar
         e) libfb303-0.9.0.jar
         f) jdo2-api-2.3-ec.jar
         g) slf4j-api-1.6.1.jar
         h) datanucleus-core-3.0.9.jar
         i) datanucleus-rdbms-3.0.8.jar
         j) commons-dbcp-1.4.jar
         k) commons-pool-1.5.4.jar
         l) antlr-runtime-3.4.jar
         m) datanucleus-api-jdo-3.0.7.jar

    HCatalog Job:
         a) above Hive required jar files
         b) hcatalog-core-0.11.0.jar
user3452265
  • 36
  • 6
0

Sometimes you may have all the jar files in your "lib" folder, but for some reason Oozie may not be reading it simply because you have a typo in your job.properties or coordinator.properties. Double check your properties file.

The "job.properties" file should read like the following - 

oozie.use.system.libpath=true
oozie.wf.application.path=${nameNode}/apps/myapp/workflow/

The "coordinator.properties" file should read like the following - 

oozie.use.system.libpath=true
oozie.coord.application.path=${nameNode}/apps/myapp/workflow/

and your "lib" folder with jars should be at -

/apps/myapp/workflow/lib
Heapify
  • 2,581
  • 17
  • 17