I need to work with the latest mahout release which is 0.9(Since I need to use all distributed recommendation algorithms). For this, which version of hadoop should I use?
In some places, it has been mentioned that hadoop 2.20 does not work well with mahout 0.9. Therefore what is the minimum hadoop version which i can use mahout 0.9.
And if I am using cloudera(CDH), what should be the version I should use to work with mahout 0.9.
The default build of all of these versions of Mahout are for Hadoop 1.x, not 2.x. So the answer is that 0.9 actually works primarily with the older Hadoop versions, and minimum version you can use it with is at most 1.x.
Later versions (from about 0.8?) have some support for building with an alternate Maven profile that should interoperate with Hadoop 2. If you like, you can always play with the build to get a release that probably works with a given Hadoop distribution's Hadoop version.
Since even CDH 4.x was based on Hadoop 2.x, the distribution of Mahout 0.7 included packaging changes that also managed to make it interoperate on Hadoop 2. Other vendors do something similar, and those changes have gone back upstream.
So the CDH distribution of each of these versions is already compatible with their matched releases of Hadoop, even where based on Hadoop 2.
CDH 5.0.0 is basically based on Hadoop 2.3. You can use Mahout 0.9 with it by targeting Hadoop release 2.3.0 in its build and then using your own build of Mahout on the cluster.
We are currently using a CDH 5.0.x cluster, but needed also some fixes and improvements of Mahout 0.9.
So we are packaging our JAR assembly with Mahout 0.9 of the CDH 5.1.Snapshot release and simple execute it via "hadoop jar" command.
Sure you can't use then the Mahout 0.9 command line interface, but at least we can write our own Jobs.
