not able to run mapreduce using luigi

Question

i am new to map-reduce jobs.May be a some basic questions but the existing documentation didn't helped me. How to run mapreduce jobs using luigi. For example wordcount_hadoop.py what are the parameters i need to pass to start a job

python examples/wordcount_hadoop.py --date-interval 2012-06-01

output:

usage: wordcount_hadoop.py [-h] [--scheduler-port SCHEDULER_PORT] [--lock]
                           [--workers WORKERS] [--lock-pid-dir LOCK_PID_DIR]
                           [--scheduler-host SCHEDULER_HOST]
                           [--local-scheduler] [--pool POOL]
                                                    {BaseHadoopJobTask,EnvironmentParamsContainer,JobTask,Task,WordCount,WrapperTask}                           ...
wordcount_hadoop.py: error: argument {BaseHadoopJobTask,EnvironmentParamsContainer,JobTask,Task,WordCount,WrapperTask}: invalid choice: '2012-07' (choose from 'JobTask', 'Task', 'WrapperTask', 'WordCount', 'EnvironmentParamsContainer', 'BaseHadoopJobTask')

score 3 · Answer 1 · answered Oct 17 '13 at 06:36

3

You need to pass in the task name in the command.

For example:

python examples/wordcount_hadoop.py WordCount --date-interval 2012-06-01

answered Oct 17 '13 at 06:36

interskh

2,511
4
20
20

not able to run mapreduce using luigi

1 Answers1