2

Mahout comes with a number of predefined Hadoop job classes (stuff derived from AbstractJob). These include jobs for computing recommendations (RecommenderJob). Mahout also defines various RecommenderEvaluators for use in testing these recommendations, but it's not clear to me how one runs these in a distributed fashion using Hadoop.

Is it: run the recommender job, and then feed the output of that into the evaluator? I looked at the RMSRecommenderEvaluatorTest, but this seems to do everythinig all in one fell swoop, and doesn't look like a Hadoop job.

Taryn
  • 242,637
  • 56
  • 362
  • 405
Mike Sokolov
  • 6,914
  • 2
  • 23
  • 31
  • Did you ever find a solution to your problem? I'm also looking at how to generate RMSE metrics from a test set. – stackoverflowuser2010 Apr 17 '14 at 22:58
  • I marked the answer correct, because I think it is. I also tried running some evaluations using precision/recall metrics, but the results were unsatisfying: I don't think my dataset was dense enough for the statistics to be meaningful. Ultimately we resorted to less formal evaluations. – Mike Sokolov Apr 23 '14 at 00:50

1 Answers1

2

Firstly I think recommendations in Mahout come in two parts: non-distributed algorithms (from Taste) and distributed hadoop jobs (ParallelAls and Item-based Collaborative Filtering). The RMSEEvaluator is in the non-distributed part, don't think it works with hadoop jobs.

For evaluating a hadoop recommendation job in Mahout, have a look at 

org.apache.mahout.cf.taste.hadoop.als.FactorizationEvaluator

You can also write some scripts and download the recommendation results to local filesystem for evaluation.

darkjh
  • 2,821
  • 7
  • 35
  • 43
  • That sounds like the same conclusion I was coming to. I'm going to poke around a little more to confirm. – Mike Sokolov May 17 '13 at 20:36
  • 1
    Just curious, can I use org.apache.mahout.cf.taste.hadoop.als.FactorizationEvaluator to evaluate the recommendation generated by ...item.RecommenderJob ? – shihpeng Apr 28 '15 at 09:01