3

I'm running into this problem for a few days. When I use hadoop1.2, it works all right. While I turn to hadoop2.x(hadoop2.4 or hadoop2.5.2), I get this problem:

java.lang.Exception: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected
    at org.apache.gora.mapreduce.GoraOutputFormat.getRecordWriter(GoraOutputFormat.java:83)
    at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:624)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:744)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)

I found that when I use hadoop2.x in the ivy.xml, it will produce a hadoop-core-1.0.1.jar automatically, it seems influenced by gora's dependency. After excluded hadoop-core-*, this problem will occur! I also update the jar file avro-mapre-1.7.6.jar to avro-mapred-1.7.6-hadoop2.jar by hand, while unfortunately nothing changes! Any ideas would be appreciated, thx!

pagid
  • 13,559
  • 11
  • 78
  • 104
wilco sheh
  • 31
  • 3
  • I solved this problem after updated gora-core-0.5.jar to gora-core-0.5.3.jar which can be found here http://search.maven.org/remotecontent?filepath=com/argonio/gora/gora-core/0.5.3/gora-core-0.5.3.jar – wilco sheh Nov 25 '14 at 07:00
  • what's more, you should update avro-mapre-1.7.6.jar to avro-mapre-1.7.6-hadoop2.jar. Another choice is to add a property to nutch-site.xml which is ` io.serializations org.apache.hadoop.io.serializer.WritableSerialization A list of serialization classes that can be used for obtaining serializers and deserializers. ` – wilco sheh Nov 25 '14 at 07:03

0 Answers0