2

Below is what I can find. But the problem is if we reuse jdbc hive session all the hive queries go as same Application-Id. Is there a way I can kill a dag?

Tez jobs can be listed using: yarn application -list

Tez jobs can be killed using: yarn application -kill Application-Id

Community
  • 1
  • 1
Himanshu Virmani
  • 2,450
  • 1
  • 24
  • 34
  • _"all the hive queries go as same"_ -- as same *what*? If you mean that your Hive session re-uses the same TEZ app master in YARN, then if you kill the app master, Hive will detect the failure, and the next query will have to restart another app master. Just what happens when the session is idle and the app master is released. – Samson Scharfrichter Mar 28 '17 at 15:22
  • Problem is if there are two dags with same app id and I have to kill one. – Himanshu Virmani Mar 28 '17 at 15:56
  • You mean you have two *running* DAGs in the same YARN job?? That sounds crazy, unless one has actually failed but TEZ did not change the status accordingly. Anyway it's the YARN job that you kill, not the DAG running inside... – Samson Scharfrichter Mar 28 '17 at 19:29
  • Disclaimer: it's been some time since I worked with TEZ (using Cloudera now). – Samson Scharfrichter Mar 28 '17 at 19:31
  • Yups two running DAGs/Hive Queries in the same AM Container/App Id. This happens when we use the same hive jdbc session to run multiple queries and tez decides to reuse the AM Container. – Himanshu Virmani Mar 29 '17 at 04:11
  • don't think you can do that even as of now, you will need 2 separate jobs and connect them as dependencies using an app like control-m. – Miadnew Nov 04 '20 at 18:57

0 Answers0