psutil's cpu_percent always returns 0.0

Question

I would like my Flask application to report how much CPU and memory it is currently using as a percentage:

import psutil
from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/test", methods=["GET"])
def healthz():
    return jsonify(msg="OK"), 200

@app.route("/stats", methods=["GET"])
def stats():
    p = psutil.Process()
    json_body = {
        "cpu_percent": p.cpu_percent(interval=None),
        "cpu_times": p.cpu_times(),
        "mem_info": p.memory_info(),
        "mem_percent": p.memory_percent()
    }
    return jsonify(json_body), 200


def main():
    app.run(host="0.0.0.0", port=8000, debug=False)

if __name__ == '__main__':
    main()

While sending a lot of requests to /test, /stats will always returns 0.0 for cpu_percent:

$ while true; do curl http://127.0.0.1:8000/test &>/dev/null; done &
$ curl http://127.0.0.1:8000/stats
{
  "cpu_percent": 0.0, 
  "cpu_times": [
    4.97, 
    1.28, 
    0.0, 
    0.0
  ], 
  "mem_info": [
    19652608, 
    243068928, 
    4292608, 
    4096, 
    0, 
    14675968, 
    0
  ], 
  "mem_percent": 1.8873787935409003
}

However, if I manually check using ipython:

import psutil
p = psutil.Process(10993)
p.cpu_percent()

This correctly returns a value greater than 0.0.

Answer 1

Simply define "p = psutil.Process()" globally (outside of stat() function). cpu_percent() keeps track of CPU times since last call, and that's how it is able to determine percentage.

The first call will always be 0.0 as calculating percentage is something which requires comparing two values over time, and as such, some time has to pass.

Answer 2

As Giampaolo pointed out, the instance of the Process needs to be at global scope because the instance tracks state to work it out based on prior call.

Do be aware though that CPU percentage can jump around quite a lot from one moment to another and especially where the time period it is calculated over keeps changing, can be quite confusing. It is perhaps better to use a background thread which works out CPU percentage average over set time ranges.

Some code I happened to have handy may be of interest:

from __future__ import print_function

import os
import time
import atexit

import threading

try:
    import Queue as queue
except ImportError:
    import queue

import psutil

_running = False
_queue = queue.Queue()
_lock = threading.Lock()

_cpu_percentage = 1800 * [0.0]
_processes = {}

def _monitor():
    global _cpu_percentage
    global _processes

    while True:
        marker = time.time()

        total = 0.0
        pids = psutil.pids()

        processes = {}

        for pid in pids:
            process = _processes.get(pid)
            if process is None:
                process = psutil.Process(pid)
            processes[pid] = process
            total += process.cpu_percent()

        _processes = processes

        _cpu_percentage.insert(0, total)

        _cpu_percentage = _cpu_percentage[:1800]

        duration = max(0.0, 1.0 - (time.time() - marker))

        try:
            return _queue.get(timeout=duration)

        except queue.Empty:
            pass

_thread = threading.Thread(target=_monitor)
_thread.setDaemon(True)

def _exiting():
    try:
        _queue.put(True)
    except Exception:
        pass
    _thread.join()

def track_changes(path):
    if not path in _files:
        _files.append(path)

def start_monitor():
    global _running
    _lock.acquire()
    if not _running:
        prefix = 'monitor (pid=%d):' % os.getpid()
        print('%s Starting CPU monitor.' % prefix)
        _running = True
        _thread.start()
        atexit.register(_exiting)
    _lock.release()

def cpu_averages():
    values = _cpu_percentage[:60]

    averages = {}

    def average(secs):
        return min(100.0, sum(values[:secs])/secs)

    averages['cpu.average.1s'] = average(1)
    averages['cpu.average.5s'] = average(5)
    averages['cpu.average.15s'] = average(15)
    averages['cpu.average.30s'] = average(30)
    averages['cpu.average.1m'] = average(60)
    averages['cpu.average.5m'] = average(300)
    averages['cpu.average.15m'] = average(900)
    averages['cpu.average.30m'] = average(1800)

    return averages

I had other stuff in this which I deleted, so hopefully what remains is still in a usable state.

To use it, add to file monitor.py and then import the module in your main and start the monitoring loop.

import monitor
monitor.start_monitor()

Then on each request call:

monitor.cpu_averages()

and extract value for time period you think makes sense.

Answer 3

The solution fo Graham seems to work but, but I found a way simpler solution, by telling it the interval, in this example it measures the last second:

psutil.cpu_percent(interval=1)

Answer 4

import psutil
from flask import Flask
app = Flask(__name__)
@app.route("/")
def index():
cpu_percent = psutil.cpu_percent()
    mem_percent = psutil.virtual_memory().percent
    Message = None
    if cpu_percent > 80 or mem_percent > 80:
        Message = "High cpu utilization detected. Please Scale up"
    return f"CPU utilization :{cpu_percent} and Memory utilization: {mem_percent}"
if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0')