2

I ran a hive query on a decently large dataset and it was taking too much time for the query so I decided to kill the application with :

    yarn kill -application-id

Now when I check from the CLI with:

    yarn application -list

then the above mentioned application does not show up in the list.

However, when I log into the Tez view from ambari the application is showing up to be still in the running state(almost been 24 hours since I created it).

I tried killing it again from the command line but it says that the application has already finished.

I also checked in the resource manager UI and the status for that job shows that it was killed.

Because of this, whenever I am trying to run any new hive job, it is just getting queued up and I am unable to run any other jobs.

Please help!

poodah nray
  • 21
  • 5

2 Answers2

1

The TEZ VIEW is an export of the application timeline server info. If you use yarn kill, hive does not properly inform the YARN Application Timeline Server that the query has been terminated. Therefore, you still see these as running in the tez view because ATS never received any update that this entered a stopped/failed state. If you are unable to run new hive jobs, it will not be related to the fact that killed applications still show as running in the tez view and you should troubleshoot that separately. The bug you described is purely cosmetic and is documented in the following places:

Jonathan
  • 121
  • 6
  • Thank you Jonathan for that information. I checked those links and even though it mentions it is only cosmetic, I am certain that it was this issue that was hogging the queue. I found a way around and am posting that as the answer. – poodah nray Jan 09 '19 at 17:20
0

So the way around I found to clear the queue so that I could run other queries was to go to /hadoop/yarn/timeline, backup the files and restart YARN. The TEZ queue was cleared up and I could start running my queries from the hive view again.

I should mention, however, that this will clear all queries(for all the users).

poodah nray
  • 21
  • 5