0

Getting ClassNotFoundException when I use the custom input format for mapreduce in hadopp using amazon EMR. The json.org dependency is present in maven pom.xml and still it throws error: 

java.lang.ClassNotFoundException: org.json.JSONException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at ERMHadoop.mywebapp.jsonFeed.JsonInputFormat.createRecordReader(JsonInputFormat.java:41)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:497)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:738)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:375)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
at org.apache.hadoop.mapred.Child.main(Child.java:249)


java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
Caused by: java.io.IOException: Task process exit with nonzero status of 255.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
CodeRocker
  • 67
  • 2
  • 3
  • 9
  • Are you using the maven assembly plugin to build a jar-with-dependencies? – Chris White Nov 27 '13 at 01:08
  • I'm not sure what is the difference between assembly plugin and normal maven. here is my pom.xml with dependencies – CodeRocker Nov 27 '13 at 01:55
  • I'm not sure what is the difference between assembly plugin and normal maven. Yes, I'm using dependencies in pom.xml to create the custom jar for emr. here is my json dependency in pom.xml org.json json 20131018 – CodeRocker Nov 27 '13 at 02:32
  • See this other related question for what i'm referring too - http://stackoverflow.com/questions/16300070/package-a-multiple-entry-jar-using-maven-for-hadoop-project – Chris White Nov 27 '13 at 11:45
  • I'm not looking for multi entry jar, I'm getting ClassNotFoundException: org.json.JSONException though the json jar is present in my manifest below: Manifest-Version: 1.0 Archiver-Version: Plexus Archiver Created-By: Apache Maven Built-By: YC31RS2 Build-Jdk: 1.6.0_39 Main-Class: ERMHadoop.mywebapp.jsonFeed.FeedDriver Class-Path: commons-lang-2.3.jar slf4j-log4j12-1.7.5.jar slf4j-api-1.7 .5.jar log4j-1.2.17.jar json-20131018.jar json-simple-1.1.jar jettiso n-1.0-alpha-1.jar stax-api-1.0.1.jar – CodeRocker Nov 27 '13 at 21:10
  • 1
    Present in the manifest yes, but is the json-simple jar present in the jar too, or on the distributed classpath? you're problem relates to a map task not being able to find this class at runtime – Chris White Nov 28 '13 at 01:20
  • Note that you can use "jar tf JAR_FILE | grep ClassName" to make sure that the JAR_FILE actually contains your class. Also make sure you did not forget to specify jar to push to your Hadoop cluster. It can be done by setJarByCass of your JobConf – Oleksandr Pryimak Jan 18 '16 at 15:11

0 Answers0