Prometheus how to expose metrics in multiprocess app with start_http_server

Question

How expose metrics in multiprocess app use start_http_server

I found many examples with gunicorn in internet but i want use start_http_server

what should i do with code below to make it work properly?

from multiprocessing import Process
import time, os
from prometheus_client import start_http_server, multiprocess, CollectorRegistry, Counter


MY_COUNTER = Counter('my_counter', 'Description of my counter')
os.environ["PROMETHEUS_MULTIPROC_DIR"] = "tmp"

def f():
    print("+1")
    MY_COUNTER.inc()

if __name__ == '__main__':
    start_http_server(8000)
    p = Process(target=f, args=())
    a = p.start()
    p2 = Process(target=f, args=())
    p2.start()
    time.sleep(1)
    print("collect")
    registry = CollectorRegistry()
    data = multiprocess.MultiProcessCollector(registry)
    while True:
        time.sleep(1)

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. — Community, Jan 26 '22 at 15:40

potato_cannon · Accepted Answer · 2022-06-17T12:48:36.710

I was figuring out the same thing, and the solution was as simple as you would imagine.

Updated your example code to work:


from multiprocessing import Process
import shutil
import time, os
from prometheus_client import start_http_server, multiprocess, CollectorRegistry, Counter


# ensure variable exists, and ensure defined folder is clean on start
prome_stats = os.environ["PROMETHEUS_MULTIPROC_DIR"]
if os.path.exists(prome_stats):
    shutil.rmtree(prome_stats)
os.mkdir(prome_stats)

MY_COUNTER = Counter('my_counter', 'Description of my counter')

def f():
    while True:
        time.sleep(1)
        print("+1")
        MY_COUNTER.inc()

if __name__ == '__main__':

    # pass the registry to server
    registry = CollectorRegistry()
    multiprocess.MultiProcessCollector(registry)
    start_http_server(8000, registry=registry)

    p = Process(target=f, args=())
    a = p.start()
    p2 = Process(target=f, args=())
    p2.start()

    print("collect")

    while True:
        time.sleep(1)

localhost:8000/metrics
# HELP my_counter_total Multiprocess metric
# TYPE my_counter_total counter
my_counter_total 16.0

Prometheus how to expose metrics in multiprocess app with start_http_server

1 Answers1