4

I'm running a few Blockchain related containers in a cloud environment (Google Compute Engine). I want to measure the power/energy consumption of the containers or the instance that I'm running.

There are tools like powerapi which is possible to do this in real infrastructure where it has access to real CPU counters. This should be possible by doing an estimation based on the CPU, Memory, and Network usage. There's one such model proposed in the literature.

Is it theoretically possible to do this? If so is there already existing tools for this task.

TharinduRanathunga
  • 221
  • 1
  • 4
  • 9
  • 3
    Remember it is more than just power consumption of the VM, it is data center Power Usage Effectiveness. Google is a leader in this area ... see: https://www.google.com/about/datacenters/efficiency/ When comparing power usage of a VM, also ask for every Watt of power consumed for processing, how much power is being absolutely consumed from the environment in order to give you that watt of power. – Kolban Apr 29 '20 at 19:14

3 Answers3

3

Measuring in cloud instances is more complicated yes. If you do have access to the underlying hardware, then it would be possible. There is a pretty recent project that allows to get power consumption metrics for a qemu/kvm hypervisor and share the metrics releveant to each virtual machine through another agent in the VM: github.com/hubblo-org/scaphandre/

They are also working (very early) on a "degraded mode" to estimate* the power consumption of the instance/vm if such communication between the hypervisor and the vm is not possible, like when working on a public cloud instance.

*: Based on the resources (cpu/ram/gpu...) consumption in the VM and characteristics of the instance and the underlying hardware power consumption "profile".

nulse
  • 862
  • 1
  • 8
  • 18
1

A generalized answer is "No, it is impossible". A program running in a virtual machine deals with virtual hardware. The goal of virtualization is to abstract running programs from physical hardware, while access to physical equipment is required to measure energy consumption. For instance, without processor affinity enabled, it's unlikely that PowerAPI will be able to collect useful statistics due to virtual CPU migrations. Time-slicing that allows to run multiple vCPUs on one physical CPU is another factor to keep in mind. Needless to say about energy consumed by RAM, I/O controllers, storage devices, etc.

A substantive answer is "No, it is impossible". Authors of the manuscript use PowerAPI libraries to collect CPU statistics, and a monitoring agent to count bytes transmitted through network. HWPC-sensor the PowerAPI relies on has distinct requirement:

"monitored node(s) run a Linux distribution that is not hosted in a virtual environment"

Also, authors emphasize that they couldn't measure absolute values of power consumption but rather used percentage to compare certain Java classes and methods in order to suppose the origin of energy leaks in CPU-intensive workloads.

As for the network I/O, the number of bytes used to estimate power consumption in their model differs significantly between the network interface exposed into the virtual guest and the host hardware port on the network with the SDN stack in between.

mebius99
  • 2,495
  • 1
  • 5
  • 9
  • I think PowerAPI library that they use is: https://github.com/abourdon/powerapi-akka not http://powerapi.org. Of course, it will be not an absolute value but why not it's not possible to do an estimation? – TharinduRanathunga May 07 '20 at 07:12
  • The last commit on GitHub for the [powerapi-akka](https://github.com/abourdon/powerapi-akka) project is dated 2016, and that is a statement that the project is deprecated. It advises to consider [powerapi.org](http://powerapi.org/) that is mentioned in the original question and explained as inapplicable to a virtual environment in the answer. – mebius99 May 07 '20 at 16:37
0

Can we conclude that , it is impossible to measure power/energy consumption of process running on virtual machines?

Balamurugan.Santhanaraj
  • 156
  • 1
  • 2
  • 8
  • I hope we can conclude that for cloud-based VMs. But if you want to measure the power of VMs managed by yourself hypervisor tools like VMWare Vshpere allow you to measure the power consumption. – TharinduRanathunga Feb 03 '21 at 17:01