I'm running a Job on hadoop. examples worked fine.
However, my job is stuck at map 100 % reduce 67 %. map reduce task status is "reduce > reduce".
my reduce() method :
public static class MyReducer extends Reducer<DoubleWritable, Text, DoubleWritable, Text> {
public void reduce(DoubleWritable key, Iterable<Text> values, Context context) throws IOException, InterruptedException {
Text res = new Text("");
for(Text value : values){
if(!value.equals("")){
res.set(res.toString() + value.toString() + " ");
}
}
context.write(key, res);
}
}
map() method :
public static class TokenizerMapper extends Mapper<Object, Text, DoubleWritable, Text>{
public void map(Object key, Text value, Context context) throws IOException, InterruptedException{
StringTokenizer input = new StringTokenizer(value.toString());
input.nextToken();
Text page = new Text(input.nextToken());
DoubleWritable vue = new DoubleWritable(Double.parseDouble(input.nextToken()) );
context.write(vue, page);
}
}
reduce task log :
2014-10-27 10:15:37,284 DEBUG [communication thread] org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 29830 29825 ]
2014-10-27 10:15:37,285 DEBUG [IPC Parameter Sending Thread #0] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 sending #614
2014-10-27 10:15:37,287 DEBUG [IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 got value #614
2014-10-27 10:15:37,288 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: Call: statusUpdate 3
2014-10-27 10:15:40,296 DEBUG [communication thread] org.apache.hadoop.yarn.util.ProcfsBasedProcessTree: [ 29830 29825 ]
2014-10-27 10:15:40,296 DEBUG [IPC Parameter Sending Thread #0] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 sending #615
2014-10-27 10:15:40,298 DEBUG [IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001] org.apache.hadoop.ipc.Client: IPC Client (592959754) connection to /xxx.xx.xx.xx:51302 from job_1414399407473_0001 got value #615
2014-10-27 10:15:40,298 DEBUG [communication thread] org.apache.hadoop.ipc.RPC: Call: statusUpdate 2