Python hadoop streaming : Setting a job name

Question

• I have a job that runs on my cluster using hadoop-streaming
• I have to start a new job for which I want to add a job name, how can I pass that option on command-line or file to setup a job name?
• In Java, you can do this by saying

JobConf conf = new JobConf(WordCount.class);
conf.setJobName("wordcount");

How can I do that with hadoop-streaming?

Answer 1

Configure the property using the -D key=value notation:

-D mapred.job.name='My Job'

You can list a whole bunch of options by calling the streaming jar with just the -info argument