0

I want to restrict number of simultaneously map tasks running at one slave node.

In my case, when I submit my job, Hadoop generate 8 map tasks, when I looked at Job history UI at port 19888, I always saw that 8 map tasks started at the same time at same slave node.

Even I tried to set this attribute mapreduce.tasktracker.map.tasks.maximum equal to 4 (from how to restrict the concurrent running map tasks?). It still didn't work for me.

Does anyone have a experiment to deal with this issue?

Community
  • 1
  • 1
nd07
  • 127
  • 9

1 Answers1

1

for hadoop 2.6 it should be newer API - see this mapred-default.xml

set mapreduce.tasktracker.map.tasks.maximum to 4

The maximum number of map tasks that will be run simultaneously by a task tracker.

Ronak Patel
  • 3,819
  • 1
  • 16
  • 29
  • `mapreduce.job.maps` >The default number of map tasks per job. Ignored when mapreduce.jobtracker.address is "local". - Do you have this set somewhere to 8? this might me overriding the max map task param – Ronak Patel Jul 07 '16 at 17:58
  • After checking, I don't set this value in the configuration, – nd07 Jul 08 '16 at 04:32