0

Using streaming, I set the number of map to 200, like this:

-D mapred.map.tasks=200 -D mapred.job.map.capacity=200

But later I found that the number of running mappers is just 9, with 500+ mapper tasks pending. This looks pretty weird to me, cuz I think there should 200 running, right? WHY just 9 running?

Alcott
  • 17,905
  • 32
  • 116
  • 173

1 Answers1

0

The number of mappers is actually controlled by the Inputformat you use.

Inputfiles are split according to the format and each split makes a map.

Some more information can be found on the Apache Wiki

Joey
  • 1,349
  • 14
  • 26
  • Well, when I submitted the job, there are 500+ mapper tasks in total, which I think should invoke 500+ mappers, right? But why the number of running mappers just 9? – Alcott Apr 16 '13 at 13:38
  • Does your system actually support 500 running mappers? It is controlled by the number of map slots per node – Joey Apr 16 '13 at 13:39