0

Hello~ I've started to study Hadoop MapReduce.

When I use cleanup method in reduce, I get a error.

The method cleanup(Reducer.Context) of type exReduce must override or implement a supertype method.

I want to use cleanup method in reducer. but It's not easy to me.

What sholud I do..?

here is my example code.

public static class exReduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable>{
        private ArrayList<ArrayList<Integer>> totalArr = new ArrayList<ArrayList<Integer>>();
        private OutputCollector<Text, IntWritable> outputCollector = null;

        @Override
        public void configure(JobConf job){
        }

        @Override
        public void reduce(Text cell, Iterator<IntWritable> measures, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException{
            if(outputCollector == null) outputCollector = output;

            ArrayList<Integer> arr = new ArrayList<Integer>();
            StringTokenizer token = new StringTokenizer(cell.toString());
            while(token.hasMoreTokens()){
                arr.add(Integer.parseInt(token.nextToken()));
            }
            totalArr.add(arr);
        }

        @Override
        protected void cleanup(Context context){
            ArrayList<ArrayList<String>> cl_data = totalArr;
            System.out.println("cl_data_size: "+cl_data.size());
            System.out.println("cl_data: ");
            for(int i = 0 ; i < cl_data.size(); i++){
                for(int j = 0; j < cl_data.get(0).size(); j++){
                    System.out.println(cl_data.get(i).get(j)+" ");
                }
                System.out.println();
            }
        }
}

I want to use this line.

extends MapReduceBase implements Reducer

not use

extends Reducer.

Is there a method to solve it ??

S.Kang
  • 581
  • 2
  • 10
  • 28
  • 1
    There are conflicts with classes in import statement. You are using deprecated/older APIs for mapreduce which provides close method for close/cleanup tasks. The latest API has cleanup method which you referring in your code. Follow to create map reduce using latest apis- https://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html – Rahul Sharma Oct 30 '16 at 19:08
  • this also will help you-http://stackoverflow.com/questions/16269922/hadoop-mapred-vs-hadoop-mapreduce – Rahul Sharma Oct 30 '16 at 19:16
  • oh..well... I didn't know that. So then I have to use import org.apache.hadoop.mapreduce.? – S.Kang Oct 31 '16 at 02:17
  • yes, follow latest mapreduce API guide. – Rahul Sharma Oct 31 '16 at 06:36
  • I appreciate your advise. have a nice day! – S.Kang Oct 31 '16 at 12:16
  • Could you answer one more question if you don't mind? I followed your advise, but I encountered a problem. I've still used the old APIs which is `MultipleTextOutputFormat` I want to use new APIs, but finding new version is not easy or understanding several code is so hard... as a result, I got a error `job.setOutputFormatClass`. what should I do...T.T – S.Kang Oct 31 '16 at 12:47
  • The new APIs inside package 'org.apache.hadoop.mapreduce'. MultipleTextOutputFormat is org.apache.hadoop.mapreduce.lib.output.MultipleOutputs in new API; – Rahul Sharma Oct 31 '16 at 16:58

0 Answers0