6

I am a newbie in MapReduce and I am trying to find a solution to a problem.I am trying to chain two map reduce jobs.The first job is getting executed but on the second job I am getting an error as follows

     INFO mapreduce.Job: Task Id : attempt_1445271708293_0055_m_000000_1, Status : FAILED
Error: java.io.IOException: Initialization of all the collectors failed.              Error in last collector was :null
    at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:414)
    at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
    at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:698)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1011)
    at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:402)
    ... 9 more
vin
  • 61
  • 1
  • 3
  • Please review my answer to your issue here https://stackoverflow.com/a/54454365/1566372 – Rady Feb 01 '19 at 06:16

2 Answers2

3

I had imported Text in Driver class as 

import com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider.Text

instead of 

import org.apache.hadoop.io.Text

because of which I was getting the error, once I rectified the mistake it started working fine.

Suraj Rao
  • 29,388
  • 11
  • 94
  • 103
Arun
  • 31
  • 2
2

I have got the same error when output types from map task didn't matched with input types of reduce task.

Last two arguments in mapper should have the same type as first two in reducer.

public class ByteCalculationMapper extends Mapper<LongWritable, Text, Text, IntWritable> {

public class ByteCalculationReducer extends Reducer<Text, IntWritable, Text, Text> {
Max Pavlov
  • 37
  • 3