2

I have added my input files 'input.txt' and 'users.txt' to HDFS successfully. I have tested Hadoop and Mahout jobs separately with success. However, when I go to run a RecommenderJob with the following command line:

bin/hadoop jar /Applications/mahout-distribution-0.9/mahout-core-0.9-job.jar org.apache.mahout.cf.taste.hadoop.item.RecommenderJob -Dmapred.input.dir=/user/valtera45/input/input.txt -Dmapred.output.dir=/user/valtera45/output --usersFile /user/valtera45/input2/users.txt --similarityClassname SIMILARITY_COOCCURRENCE

This is the output I get:

Exception in thread "main" java.io.IOException: Cannot open filename /user/valtera45/temp/preparePreferenceMatrix/numUsers.bin at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1444) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.(DFSClient.java:1435) at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:347) at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:178) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:351) at org.apache.mahout.common.HadoopUtil.readInt(HadoopUtil.java:339) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.run(RecommenderJob.java:172) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.mahout.cf.taste.hadoop.item.RecommenderJob.main(RecommenderJob.java:322) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

Whenever I run a standalone Mahout job, a temp folder gets created within the Mahout directory. The RecommenderJob can't seem to get past this step. Any ideas? Thanks in advance. I know the input files I am using are well formatted because they have worked successfully for others.

valtera45
  • 21
  • 2
  • Thanks, Ramanan. I tried setting permissions for every HDFS and Hadoop temp folder I could find and I still got the same error. Even tried on every node in my cluster. – valtera45 Jul 21 '14 at 23:23
  • Check if this helps: http://www.techques.com/question/1-9532538/Trouble-running-RecommenderJob-on-hadoop – Ramanan Jul 22 '14 at 05:12
  • Thanks a lot. No change in the output, though. I've tried different input all of which I know is valid because I've seen it run on different machines. – valtera45 Jul 23 '14 at 02:46
  • @RamananR I think the link you shared is broken. Can you or valtera45 be kind to explain what the solution was. I am facing the same problem – Caffeinated Coder Feb 24 '16 at 23:29

1 Answers1

0
hadoop jar mahout-core-0.8-job.jar org.apache.mahout.cf.taste.hadoop.item.RecommenderJob -Dmapred.input.dir=large_data.csv -Dmapred.output.dir=output/output1.csv -s SIMILARITY_LOGLIKELIHOOD --booleanData --numRecommendations 5

I am using this and my program is running successfully on ec2 instance with mahout and hadoop but i am not able to get relevant results. if anyone knows anything about it please revert on this.

Meenesh Jain
  • 2,532
  • 2
  • 19
  • 29
user5233842
  • 1