wordcount from eclipse

Question

I was using the eclipse plugin for hadoop. I can see all the files in HDFS by making a hadoop server but when I try to run the wordcount.java file from the eclipse, it gives me exception whereas from the terminal it runs smoothly. The exception is below.

2/11/14 04:09:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for  your platform... using builtin-java classes where applicable
12/11/14 04:09:06 WARN mapred.JobClient: No job jar file set.  User classes may not be     found. See JobConf(Class) or JobConf#setJar(String).
12/11/14 04:09:06 WARN snappy.LoadSnappy: Snappy native library not loaded
12/11/14 04:09:06 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-hduser/mapred/staging/hduser1728681403/.staging/job_local_0001
12/11/14 04:09:06 ERROR security.UserGroupInformation: PriviledgedActionException as:hduser cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/user/hduser/gutenberg
Exception in thread "main" org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/user/hduser/gutenberg
    at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:197)
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208)
    at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:989)
    at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:981)
    at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1261)
    at WordCount.run(WordCount.java:149)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at WordCount.main(WordCount.java:155)

score 2 · Accepted Answer · answered Nov 14 '12 at 12:45

2

I'd start with investigating this:

ERROR security.UserGroupInformation: PriviledgedActionException as:hduser cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/user/hduser/gutenberg

It seems it causes the problem. Are you sure this is the proper path? If so, you may not have the privilage to access to it. Later on I would try to eliminate as much WARN as I can.

answered Nov 14 '12 at 12:45

Bela Vizer

2,527
22
26

Hi thanks for replying. The problem is the input path. When I give input like /user/files/.., it doesnt work but when I input it with the direct url i.e. hdfs://localhost:54310/user/... it works. The error i get is the input path not existing, but it should say like hdfs://localhost:9000/... and my exception says like file://... not found. Any ideas. Thanks again – shujaat Nov 15 '12 at 04:05

score 1 · Answer 2 · answered Oct 18 '13 at 17:30

1

Thanks Shujaat that solved my problem. From Eclipse I was getting the same issue... use hdfs://localhost:54310/user/... instead of "/user/..."

answered Oct 18 '13 at 17:30

Dutta

663
2
11
30

wordcount from eclipse

2 Answers2