9

I am trying to create a mapper only job via AWS (a streaming job). The reducer field is required, so I am giving a dummy executable, and adding -jobconf mapred.map.tasks=0 to the Extra Args box. In the hadoop environment (version 0.20) I've installed, no reducer jobs will launch, but in AWS the dummy executable launches and fails.

How can run a no reducer/ mapper only jobs in AWS?

Paul Bellora
  • 54,340
  • 18
  • 130
  • 181
sagie
  • 2,998
  • 3
  • 22
  • 31

2 Answers2

11

you can also use cat or NONE as reducer argument.

0

There is no way to create a mapper only job in amazon M/R. I am using the Identity reducer (org.apache.hadoop.mapred.lib.IdentityReducer) instead.

sagie
  • 2,998
  • 3
  • 22
  • 31