0

I have a Hadoop Map Reduce job where I have split the input using a line reader.

Map input records=10.

Is it possible to partition the map tasks output based on the LongWritable key that identifies the line reader split ?

If not - Is there another way to partition the output for each map task ?

Chris MacKenzie
  • 1
  • 1
  • Can you confirm that you need to partition map output in such a way that whole key-value pair coming under a single Map task should go to a single reduce task(one-one mapping b/w Map and Reduce tasks)? – donut Jun 26 '14 at 16:49
  • Thats correct @donut. I'm using Hadoop 2.4 in local mode. I have a partitioner class which is testing the key from the map class. The key from the map class is generated by a the line reader and is basically the start point of each new record. I've tried debugging in eclipse with a breakpoint in the partitioner but the partitioner is not being called. – Chris MacKenzie Jun 27 '14 at 04:11

0 Answers0