0

Error: java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:320) at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:533) at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:130) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:34) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)

Ramesh Thangaraj
  • 21
  • 4
  • Sys.setenv("HADOOP_CMD"="/home/hduser/hadoop/bin/hadoop") Sys.setenv("HADOOP_CONF"="/home/hduser/hadoop/etc/hadoop") Sys.setenv("HADOOP_STREAMING"="/home/hduser/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.4.1.jar") library(rhdfs) hdfs.init() library(rmr2) wc.map <- function(k,v){ words.list = unlist(strsplit(v,'\\s')) return (keyval(words.list,1))} Reduce Function:wc.reduce <- function(k,v){ return(keyval(k,sum(v))) } out <- mapreduce(input="/user/hduser/wordcount/wordcount_input.txt", output="/user/hduser/wordcount/output",input.format="text",map=wc.map,reduce=wc.reduce) – Ramesh Thangaraj Dec 03 '16 at 23:05
  • What RHadoop Map reduce job? See [How to create a Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve). – MikeJRamsey56 Dec 04 '16 at 02:57
  • It's a map reduce job to determine wordcount of a text file.I am getting error in hadoop streaming. Not sure what the issue is ? – Ramesh Thangaraj Dec 04 '16 at 07:07

0 Answers0