0

I have written several RHadoop programs that work even though they return warnings such as:

Converting to.dfs argument to keyval with a NULL key

when inputting data with to.dfs.

However, some programs fail fatally with no warnings other than

Converting to.dfs argument to keyval with a NULL key

followed by

ERROR streaming.StreamJob: Job not successful. Error: # of failed Map Tasks exceeded allowed limit. FailedCount: 1.

Is the NULL key warning normally associated with failed Map tasks?

I know the standard advice is to look at stderr, but the stderr for the failed job is empty! Zero lines, zero characters.

dataquerent
  • 267
  • 2
  • 5
  • 13

2 Answers2

1

As far as I know, the

Converting to.dfs argument to keyval with a NULL key

Is an usual warning, and it doesn't make the job fail.

Did you try fetching your data stored with the to.dfs command using a from.dfs command to see if it worked ? If it does, the problem probably lies eslewhere.

kmoulart
  • 26
  • 4
  • Yes, I get the NULL warning even for the jobs that work correctly. But I have a lot of jobs that fail, so I was wondering whether the NULL had anything to do with it. Thanks. – dataquerent Dec 20 '13 at 01:49
1

i think this issue is of streaming ,i used this line in my R terminal when this problem come it may help you Sys.setenv(HADOOP_HOME='/usr/lib/hadoop-0.20-mapreduce') Sys.setenv(HADOOP_CMD='/usr/bin/hadoop') Sys.setenv( HADOOP_STREAMING='/usr/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.1.1.jar')

streaming may be differ by your jar file .like /usr/lib/hadoop/contrib/streaming/hadoop-streaming-1.0.2.jar is my jar...you can check by your path.

user3392328
  • 49
  • 1
  • 1
  • 9
  • Lately I have been using: `HADOOP_CMD=/home/user/hadoop-1.2.1/bin/hadoop HADOOP_STREAMING=/home/user/hadoop-1.2.1/contrib/streaming/hadoop-streaming-1.2.1.jar` Thanks for your mention of the HADOOP_HOME variable. – dataquerent Mar 12 '14 at 01:26