0

I am running an example mapreduce job that comes with Hadoop 2.8.1

I am using these commands:

bin/hdfs dfs -copyFromLocal etc/hadoop/core-site.xml .

bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep ./core-site.xml output ‘configuration’

However, when I run this, the job exits (the main error appears in the title):

***********s-mbp-2:hadoop-2.8.1 ***********$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.1.jar grep ./core-site.xml output ‘configuration’ 
17/09/12 14:30:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/09/12 14:30:29 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/09/12 14:30:29 INFO input.FileInputFormat: Total input files to process : 1
17/09/12 14:30:30 INFO mapreduce.JobSubmitter: number of splits:1
17/09/12 14:30:30 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1505251091124_0001
17/09/12 14:30:30 INFO impl.YarnClientImpl: Submitted application application_1505251091124_0001
17/09/12 14:30:30 INFO mapreduce.Job: The url to track the job: http://***********s-mbp-2.lan:8088/proxy/application_1505251091124_0001/
17/09/12 14:30:30 INFO mapreduce.Job: Running job: job_1505251091124_0001
17/09/12 14:30:33 INFO mapreduce.Job: Job job_1505251091124_0001 running in uber mode : false
17/09/12 14:30:33 INFO mapreduce.Job:  map 0% reduce 0%
17/09/12 14:30:33 INFO mapreduce.Job: Job job_1505251091124_0001 failed with state FAILED due to: Application application_1505251091124_0001 failed 2 times due to AM Container for appattempt_1505251091124_0001_000002 exited with  exitCode: 127
Failing this attempt.Diagnostics: Exception from container-launch.
Container id: container_1505251091124_0001_02_000001
Exit code: 127
Stack trace: ExitCodeException exitCode=127: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:972)
    at org.apache.hadoop.util.Shell.run(Shell.java:869)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1170)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:236)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:305)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:84)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)


Container exited with a non-zero exit code 127
For more detailed output, check the application tracking page: http://***********s-mbp-2.lan:8088/cluster/app/application_1505251091124_0001 Then click on links to logs of each attempt.
. Failing the application.
17/09/12 14:30:33 INFO mapreduce.Job: Counters: 0
17/09/12 14:30:33 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/09/12 14:30:33 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/***********/.staging/job_1505251091124_0002
org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://localhost:9000/user/***********/grep-temp-576807334
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:329)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:271)
    at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59)
    at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:393)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:303)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:320)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:198)
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1341)
    at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1338)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1338)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1359)
    at org.apache.hadoop.examples.Grep.run(Grep.java:94)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
    at org.apache.hadoop.examples.Grep.main(Grep.java:103)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
    at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
    at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:148)

What is going wrong, and how do I fix it?

halfer
  • 19,824
  • 17
  • 99
  • 186
user1072337
  • 12,615
  • 37
  • 116
  • 195

1 Answers1

0

It looks like the program can't access the path in the exception. You may try to manually create that directory with mkdir in hdfs system before running the code. Hope it will help.

mgokhanbakal
  • 1,679
  • 1
  • 20
  • 26