8

Are the “resource manager” and the “hdfs namenode” always installed on the same host?

1) When I want to send an http request (YARN REST API) to get new application id I am using this web uri:

http://<rm http address:port>/ws/v1/cluster/apps/new-application

This port is the Resource-Manager Web UI http port, it’s default value is 8088 as shown in img1: source for img1Yarn Ports

2) When I want to send an http request (WebHDFS REST API) for hdfs commands, for example to get file status I am using this web uri:

http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=GETFILESTATUS

This <PORT> is the NameNode WebUI http Port, it’s default value is 50070 as shown in img2: source for img2 HDFS Ports

Are the hosts of those components (ResourceManager and NaneNode) are always installed on the same host?

Any help would be appreciated, Thanks!

Xquery
  • 187
  • 1
  • 3
  • 9
  • 2
    You can install the YARN Resource Manager and the HDFS Namenode to any host you like. They don't need to share the host. – vanekjar Apr 01 '15 at 09:43

1 Answers1

11

It is not necessary to run resourceManager and nameNode on a single machines, we have such configuration when working in a single node/small cluster---with few nodes---configuration. When there is large cluster, master nodes usually run resourceManager, nameNode and secondaryNamenode on different machines.

cluster topology

Have a look at these links:

Master Nodes in Hadoop Clusters

HortonWorks: Typical Hadoop Cluster

Community
  • 1
  • 1
Hamza Zafar
  • 1,320
  • 12
  • 17
  • The note says that Datanodes, NodeManagers, regionservers are co-deployed. Do you mean that each worker node has a datanode, a nodemanager and a regionserver ?! – Yassir S Mar 24 '17 at 09:30