2

I'm running a Job on hadoop. examples worked fine.

However, my job is stuck at map 100 % reduce 67 %. map reduce task status is "reduce > reduce".

my reduce() method :

public static class MyReducer extends Reducer<DoubleWritable, Text, DoubleWritable, Text> {

    public void reduce(DoubleWritable key, Iterable<Text> values, Context context) throws IOException, InterruptedException {
        Text res = new Text("");

        for(Text value : values){
            if(!value.equals("")){
                res.set(res.toString() + value.toString() + " ");
            }
        }
        context.write(key, res);
    }
}

map() method :

public static class TokenizerMapper extends Mapper<Object, Text, DoubleWritable, Text>{

    public void map(Object key, Text value, Context context) throws IOException, InterruptedException{
         StringTokenizer input = new StringTokenizer(value.toString());

         input.nextToken();
         Text page = new Text(input.nextToken());

         DoubleWritable vue = new DoubleWritable(Double.parseDouble(input.nextToken()) );
         context.write(vue, page);
    }
}

reduce task log :

2014-10-27 10:15:37,284 DEBUG [communication thread] org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 29830 29825 ]
2014-10-27 10:15:37,285 DEBUG [IPC Parameter Sending Thread #0] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 sending #614
2014-10-27 10:15:37,287 DEBUG [IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 got value #614
2014-10-27 10:15:37,288 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: Call: statusUpdate 3
2014-10-27 10:15:40,296 DEBUG [communication thread] org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 29830 29825 ]
2014-10-27 10:15:40,296 DEBUG [IPC Parameter Sending Thread #0] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 sending #615
2014-10-27 10:15:40,298 DEBUG [IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 got value #615
2014-10-27 10:15:40,298 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: Call: statusUpdate 2
Ludovic S
  • 185
  • 1
  • 2
  • 9

0 Answers0