How to use ResourceManager REST API when ResourceManager is HA enabled?

Question

We have been using the ResourceManager (RM) REST APIs (Apache Hadoop 2.7.x) to fetch different details like running application details, scheduler info, etc in our Java code.

For example, the below request gives the scheduler details:

GET http://rm_http_address:port/ws/v1/cluster/scheduler

But how can my Java code get the above details if the RM is in HA mode and the code is not sure which host is the active RM?

Please help.

score 1 · Answer 1 · answered Aug 16 '17 at 18:51

1

You shouldn't have to do anything because the RMs should redirect automatically. It was broken for a while but YARN-2605 shows it fixed in 2.7.1. However, you might still hit YARN-4820 (which was fixed in 2.8.0 but not 2.7.1) if you are calling the APIs with query parameters.

https://issues.apache.org/jira/browse/YARN-2605 - [RM HA] Rest api endpoints doing redirect incorrectly
https://issues.apache.org/jira/browse/YARN-4820 - ResourceManager web redirects in HA mode drops query parameters

answered Aug 16 '17 at 18:51

tk421

5,775
6
23
34

Thanks for answer. But I have already mentioned in the bounty "Currently, the redirection internally works but makes the calls quite slow." – Rakesh Rakshit Aug 18 '17 at 13:20
Can you update your question? I can't comment on "quite slow" when the only detail you've provided is "Java code". – tk421 Aug 18 '17 at 16:34
By Java code I mean any HttpClient API which allows you to fire a GET request. – Rakesh Rakshit Aug 21 '17 at 12:28
And the response time of the GET call increases in HA mode – Rakesh Rakshit Aug 21 '17 at 12:29
For my setup, talking to the stand-by RM, only takes 0.10-0.23 sec longer than talking to the active RM. I don't see this as a significant slow down. – tk421 Aug 21 '17 at 20:55

How to use ResourceManager REST API when ResourceManager is HA enabled?

1 Answers1