Control intermediates results in hadoop

Question

I want to take control of intermediate results between Map and Reduce with hadoop. I would want to specify where copy these results after the Map. I would to choose data which will be reduced. In summary i want map's results before process shuffle and sort and do what i want. If you have a solution please tell it me.

Thanks

Answer 1

You could process data after your particular mapper get InputSplit. Specify logic into map function.

1. I would want to specify where copy these results after the Map

   use Context Java Class and FileSystem to flush results on FS (local, hdfs, ftp, ...)

2. I would to choose data which will be reduced

   i want map's results before process shuffle and sort and do what i want

   specify logic on your map function in Mapper Class