In Hadoop, the mapper and reducer classes should extend the Mapper or Reducer interface. However, I could not find the interface which the combiner class should extend. What is the signature for the combiner class in Hadoop?
Asked
Active
Viewed 2,049 times
2 Answers
0
A combiner extends the Reducer interface, and its signature should emit the same Key / Value types as it consumes, for example the Word Count combiner has the following signatures (for the new o.a.h.mapreduce and old o.a.h.mapred packages):
public class MyCombinerNewApi
extends Reducer<Text, IntWritable, Text, IntWritable> {
}
public class MyCombinerOldApi
implements Reducer<Text, IntWritable, Text, IntWritable> {
}
It is sometimes common for the Combiner class to be the same as the Reducer class (as in the Word Count example).
Some nice links explaining Combiners in more detail:
- Phillipe Adjimun's blog on "To use or not to use a combiner"
- Pro Hadoop Book - Combiners
Chris White
- 29,949
- 4
- 71
- 93
-
+1 , I found another small but good article on combiners here : http://bigdataspeak.wordpress.com/2013/01/05/what-are-combiners-in-hadoop/ – Jan 05 '13 at 05:31
0
Combiners is mini- Reducer . They follow the same signature of Reducer itself
public class Combiner extends Reducer<Text, IntWritable, Text, IntWritable> {
USB
- 6,019
- 15
- 62
- 93