Hadoop Yarn Daemons Stop Running

Question

Running start-dfs.sh and start-yarn.sh followed by jps gives the following output:

8227 ResourceManager
8328 NodeManager
7804 NameNode
8366 Jps
8077 SecondaryNameNode
7901 DataNode

If I run jps about 5 seconds later, the ouput is:

7804 NameNode
8473 Jps
8077 SecondaryNameNode
7901 DataNode

The two yarn daemons (ResourceManager and NodeManager) stop working for some reasons. Any idea as to why this may happen?

Like jkbkot said below, check the logs, usually the reasons are obvious. — Vishal, Dec 26 '13 at 04:40

score 2 · Accepted Answer · answered Dec 25 '13 at 19:45

2

This typically happens when there are problems starting up the services. It could be a lot of things - missing configuration, permission issues, etc.

You'll have to inspect the logs of ResourceManager and NodeManager for more details.

answered Dec 25 '13 at 19:45

Jakub Kotowski

Yea, it was alot of different things. Specifically, I had to modify the yarn-site.xml and mapred-site.xml config files. – Olshansky Dec 26 '13 at 05:29
@Olshansk could you mention which properties you had to modify? – Alexey Grigorev Oct 26 '15 at 08:55
Unfortunately this was so long ago that I really can't remember. If I recall, it had to do with the total amount of memory I was allocating for the whole cluster, combined with the number of mappers / reducers, and the min/max memory for each mapper and reducer. Let me know if you're still stuck. I can probably dig this project up and look and my git history. – Olshansky Oct 28 '15 at 04:09

1 Answers1