1

I recently upgraded HDP from 2.6.5 to 3.1.0, which runs YARN 3.1.0, and I can no longer kill applications from the YARN ResourceManager UI, using either the old (:8088/cluster/apps) or new (:8088/ui2/index.html#/yarn-apps/apps) version. I can still kill them using the shell in RHEL 7 with yarn app -kill {app-id}

These applications are submitted via Livy. Here is my workflow:

Open the ResourceManagerUI, open the Application, click Settings and choose Kill Application. Notice, the 'Logged in as:' is set to UNKNOWN_USER:

step1

Confirm that I want to kill the Application:

step2

I get the following error in the UI:

step3

Opening the console in Chrome, I see a 401 (Unauthorized) error.

step4

If I try this from the old UI I am able to expand the error message and it shows the following:

{"RemoteException":{"exception":"AuthorizationException","message":"Unable to obtain user name, user not authenticated","javaClassName":"org.apache.hadoop.security.authorize.AuthorizationException"}}

I've read lots of posts, verified and changed several settings to try to fix this with no luck. Here are some of the settings I checked or changed as a result of my research:

hadoop.http.filter.initializers=org.apache.hadoop.security.HttpCrossOriginFilterInitializer,org.apache.hadoop.http.lib.StaticUserWebFilter
hbase.security.authentication=simple
hbase.security.authorization=false
yarn.nodemanager.webapp.cross-origin.enabled=true
yarn.resourcemanager.webapp.cross-origin.enabled=true
yarn.resourcemanager.webapp.delegation-token-auth-filter.enabled=false
yarn.resourcemanager.webapp.ui-actions.enabled=true
yarn.timeline-service.http-authentication.simple.anonymous.allowed=true
yarn.timeline-service.http-authentication.type=simple
yarn.webapp.api-service.enable=true
yarn.webapp.ui2.enable=true
ranger.add-yarn-authorization=false

Some of these seem way off base to me, like the hbase stuff, since I don't think that has anything to do with what I'm seeing. However, some users, in other situations, had it work for them so I wanted to try it.

Digging through the documentation it seems like you need to be authenticated before you can call the API. However, that same language was in the documentation for 2.6.5, which is the version of YARN I was running before where this worked.

Hopefully someone can point me to documentation that more clearly outlines what I can do to resolve the issue.

Thanks in advance.

ammills01
  • 709
  • 6
  • 28
  • New tag request - yarn-resourcemanager - This would be of benefit since the current resourcemanager tag is being used for multiple different resource managers. – ammills01 Feb 14 '19 at 21:53

1 Answers1

0

Hey I know this isn't solution (I'm experiencing the same issue post-upgrade), but I found adding "?user.name=" at the end of the url to the old resource manager url will log you in as that user on both pages. I've found the old RM page as the only way to kill jobs though.

Rick
  • 15
  • 7
  • Thanks for the workaround suggestion Rick. When I do that I get an error: HTTP ERROR 500 Problem accessing /cluster. Reason: Server Error Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&' at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:91) – ammills01 Dec 05 '19 at 20:53
  • 1
    `?user.name=yarn` whatever the superuser was before. Let me know if that works. – Rick Dec 09 '19 at 16:40