0

I'm now practicing MapReduce in Hadoop, and I got this code of matrix multiplication, there isn't any question running it, but the output format is not what I want. I currently have:

00    66
01    78
02    90

The format I want is like 

0,0,66
0,1,78
0,2,90

How do I change the code?

Binary Nerd
  • 13,872
  • 4
  • 42
  • 44
Ren
  • 5
  • 4
  • You need to change your keys format and add a comma and see this answer to change the delimiter between the key/value - http://stackoverflow.com/questions/11031785/hadoop-key-and-value-are-tab-separated-in-the-output-file-how-to-do-it-semicol – Binary Nerd Mar 28 '17 at 18:35
  • @BinaryNerd Actually I just can't find where to change the key format..., and I have tried mapred.textoutputformat.separator", "," , mapreduce.textoutputformat.separator", ",", and mapreduce.output.textoutputformat.separator", ",", but none of them work. – Ren Mar 28 '17 at 18:42
  • So you fixed the delimiter and its just the key that's the problem now? – Binary Nerd Mar 28 '17 at 18:43
  • @BinaryNerd no, delimiter isn't fixed either, maybe i should try conf.set("mapreduce.output.key.field.separator", separator); conf.set("mapred.textoutputformat.separatorText", separator); to see if it can fix it p.s. my hadoop ver. is 2.6 – Ren Mar 28 '17 at 18:46
  • You need to add `conf.set("mapreduce.textoutputformat.separator", ",");` to your main, as stated in the answer i linked to. – Binary Nerd Mar 28 '17 at 18:49
  • I've tried `conf.set("mapred.textoutputformat.separator", ","); conf.set("mapreduce.textoutputformat.separator", ","); conf.set("mapreduce.output.textoutputformat.separator", separator); conf.set("mapreduce.output.key.field.separator", separator);` But none of them work :( – Ren Mar 28 '17 at 18:54
  • @BinaryNerd I fixed the delimiter, it seems that code has to be added before `FileSystem fs = FileSystem.get(conf);`, or `Job job = Job.getInstance(conf);` And for my version(which is 2.6), seems only `conf.set("mapreduce.output.textoutputformat.separator", ",");` work, But I still can't figure out how to change key problem – Ren Mar 28 '17 at 19:08

1 Answers1

0

You can change the format of the key in the Mapper. There are two places where it creates sKey. At the moment both lines (56 & 68) that look like:

sKey = row+i; (taking line 56 as an example)

So you can just change it too:

sKey = row+","+i;

Which will give you the format you're after.

Binary Nerd
  • 13,872
  • 4
  • 42
  • 44