0

I am running map reduce jobs in a kerberos secured hadoop cluster.

Distribution is Hortonworks.

Some jobs that were working fine before, happen to fail since we set kerberos.

We reach the ulimit -u (nbproc) for the applicative user that launches the hadoop client.

It seems that some processes (YarnChild, MrAppMaster) that were running with yarn user now run with the applicative user that launches the hadoop client.

We need to increase the ulimit -u for the applicative user, but what value can we set ?

Thx for help

rgirodon
  • 1
  • Naïve approach: looks like you were able to run all the jobs as `yarn` "system" account, therefore the `ulimits` for `yarn` would seem to be a good starting point to set up your app service account. – Samson Scharfrichter May 13 '16 at 14:08
  • Thanks @SamsonScharfrichter for your answer. Indeed, some other processes remain launched by yarn user. We increased our ulimit for our applicative user after a few benches, and things seem to be ok now. – rgirodon Jun 03 '16 at 07:21

0 Answers0