3

Hive logs contain information of the total time taken by the query, the number of rows fetched, etc along with the internal method calls.

Is there any way to get cpu, IO and memory usage for a particular hive query?

Dev
  • 13,492
  • 19
  • 81
  • 174

1 Answers1

0

[UPDATE]

If you want to check the cpu usage and memory for a particular query, then you can use hive JMX based approach to collect some of those details like:

https://community.hortonworks.com/articles/62211/enabling-jmx-monitoring-for-hiveserver2.html

In case you're using Ambari HDP, then you can use Grafana services also. Grafana also can provide many graphs related to Hive:

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.2.0/bk_ambari-operations/content/llap_overview.html

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.2.0/bk_ambari-operations/content/grafana_hive_hiveserver2.html

In case you want to know the CPU utilization by Database then you can follow this link:

CPU utilization by database?

[Previous Answer]

You can use ip-addr:8088 (e.g: 192.168.118.111:8088) and there you can check how many containers were created and how much cpu + memory was required by the job to be executed successfully.

Abhinav
  • 658
  • 1
  • 9
  • 27
  • Thanks, Abhinav, yes I understand hadoop map-reduce logs can be helpful. But I was wondering if its possible with HiveServer2 logs. – Dev May 11 '18 at 12:50
  • Nope. I was looking for HiveServer2 matrices or other native hive based approach than parsing map-reduce logs. – Dev May 16 '18 at 08:37