How to get active namenode hostname from Cloudera Manager REST API?

Question

I'm able to access the Cloudera manager rest API.

curl -u username:password http://cmhost:port/api/v10/clusters/clusterName

How to find the active namenode and resource mangarer hostname?

I couldn't find anything relevant from API docs.

http://cloudera.github.io/cm_api/apidocs/v10/index.html

Note: Cluster is configured with high availability

Answer 1

You need to use this endpoint:

http://cloudera.github.io/cm_api/apidocs/v10/path__clusters_-clusterName-services-serviceName-roles-roleName-.html

Then do the following:

For each Name Node:

$ curl -u username:password \
http://cmhost:port/api/v10/clusters/CLNAME/services/HDFS/roles/NN_NAME

Replacing:

• CLNAME with your clusterName
• HDFS with your HDFS serviceName
• NN_NAME with your NameNode name

This will return the apiRole object which has a field called haStatus. The one that shows "ACTIVE" is the active NameNode.

For the Resource Manager do similar steps:

For each Resource Manager:

$ curl -u username:password \
http://cmhost:port/api/v10/clusters/CLNAME/services/YARN/roles/RM_NAME

Where:

• YARN with your YARN serviceName
• RM_NAME with your Resource Manager name

Once you have the right NameNode and Resource Manager, use:

http://cloudera.github.io/cm_api/apidocs/v10/path__hosts_-hostId-.html

to map the hostId to the hostname.

Answer 2

You can get a bulk of HDFS related information for hosts by using the REST API:

$ python build.py username:password cmhost:port
$ cat build.py
import sys
import json
import requests

args = sys.argv
if len(args) != 3:
  print "Usage: python %s login:password host:port" % args[0]
  exit(1)

LP = args[1]
CM = args[2]

host = {}
hosts = requests.get('http://'+LP+'@'+CM+'/api/v10/hosts').json()
for h in hosts['items']:
  host[h['hostId']] = h['hostname']

nameservices = requests.get('http://'+LP+'@'+CM+'/api/v10/clusters/cluster/services/hdfs/nameservices').json()
for ns in nameservices['items']:
  print('hdfs.NS:' + ns['name'])

services = requests.get('http://'+LP+'@'+CM+'/api/v10/clusters/cluster/services').json()
for s in services['items']:
  if (s['name'] == 'hdfs'):
    roles = requests.get('http://'+LP+'@'+CM+'/api/v10/clusters/cluster/services/' + s['name'] + '/roles').json()
    srv = {}
    for r in roles['items']:
      suff = '.' + r.get('haStatus') if r.get('haStatus') else ''
      key = s['name'] + '.' + r['type'] + suff
      srv[key] = srv.get(key) + ',' + host[r['hostRef']['hostId']] if srv.get(key) else host[r['hostRef']['hostId']]
    for s in srv:
      print(s + ":" + ','.join(sorted(srv[s].split(','))))

Then you'll get something like this, just grep for hdfs.NAMENODE.ACTIVE (or slightly change the python script):

hdfs.NS:H1
hdfs.NAMENODE.ACTIVE:h6
hdfs.NAMENODE.STANDBY:h1
hdfs.FAILOVERCONTROLLER:h1,h2,h3
hdfs.DATANODE:h1
hdfs.HTTPFS:h1,h2,h3
hdfs.GATEWAY:h1,h2,h3
hdfs.JOURNALNODE:h4,h5
hdfs.BALANCER:h7