0

Good morning,

I did not succeed to find a mapReduce example on YARN (i.e. The 2nd version of MapReduce), the one that is always presented is WordCount which is just the very same code as the one presented on the first version of MapReduce. Even "Hadoop: the definitive guide" doesn't have codes in YARN!

Can you provide me with a code that shows me the difference of writing a mapReduce code in the previous version and the newest version?

In fact, I was trying to write a branch and bound code on MR1 but then I saw that YARN can make the things easiser thanks to BranchReduce.

Any help is appreciated, Thanks in advance

Hadoop User
  • 114
  • 1
  • 10

1 Answers1

0

You could compile a program written for MRv1 with Yarn(MRv2) without modifying any single line of the source code. It is completely source-code compatible.

Here is the Yarn Example: http://wiki.apache.org/hadoop/WordCount

Here is the Map Red 1 Example: https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html#Example%3A+WordCount+v1.0

Some obvious differences to note at the API level:
- New API uses abstract classes over interfaces
- The mapreduce package is different

Jasper
  • 8,440
  • 31
  • 92
  • 133
  • Thanks a lot for your answer, but what about JobTracker?TaskTracker etc? I thought that we should replace them by ResourceManager and NodeManager !! would I be always have the access to the JobTracker localhost? – Hadoop User Feb 26 '14 at 10:32
  • Yes ofcourse with Yarn you need Resource Manager. Node Manager - the ecosystem is slightly different: http://hadoop.apache.org/docs/current2/hadoop-yarn/hadoop-yarn-site/YARN.html – Jasper Feb 26 '14 at 11:56
  • So, does that mean that the program written for MRv1 will be implicitly changeed to be matched with the structure of YARN? thanks a lot. – Hadoop User Feb 26 '14 at 12:54
  • You can run the program written for MRv1 unchanged in YARN - Yes. You just need to recompile. Backward compatibility exists! – Jasper Feb 26 '14 at 13:15
  • OK thanks. By the way, do you know about BranchReduce? how can I use this library in Yarn? – Hadoop User Feb 26 '14 at 13:26
  • Branch Reduce? That's cloudera specific i guess.. sorry don't know really know abt it much – Jasper Feb 26 '14 at 15:20