Get the name of celery worker from inside a celery task?

Question

I'd like a celery task to be able to get the name of the worker executing it, for logging purposes. I need to handle this from within the task, rather than querying the broker directly. Is there a way to do this? I'm using celery with RabbitMQ, if that matters.

Please set my solution to resolved if it worked for you :) – Roy Iacob May 26 '14 at 05:55 — Roy Iacob, May 26 '14 at 05:55

Teo Sibileau · Accepted Answer · 2017-04-26T19:45:33.553

9

Use the celeryd_after_setup signal to capture the worker name like this:

from celery.signals import celeryd_after_setup

@celeryd_after_setup.connect
def capture_worker_name(sender, instance, **kwargs):
    os.environ["WORKER_NAME"] = '{0}'.format(sender)

edited Apr 26 '17 at 19:45

answered Apr 26 '17 at 18:04

Teo Sibileau

294
2
5

1

This should be the accepted answer, however I would put the name inside the celery config, and not in an environment variable – Mart10 Dec 20 '18 at 15:19
1

If you're using the `celery_worker` fixture to test, use the `worker_init` signal instead as `celeryd_after_setup` is not called in test. – Sirupsen Jan 20 '22 at 22:49

Roy Iacob · Answer 2 · 2014-05-28T15:37:30.533

7

You need to utilize billiard which holds the workers:

from celery import task
from billiard import current_process

@task
def getName():
    p = current_process()
    return p.index

Then make a global dictionary that maps ids->names on process creation.

edited May 28 '14 at 15:37

answered May 25 '14 at 15:42

Roy Iacob

412
3
13

3

This isn't quite what I was looking for. When you start a celery worker, you can give it a name (-n flag). I'm wondering if there is a way to get that name from within the task. – imichaeldotorg May 26 '14 at 17:46
1

not quite as straight forward but why not make a global dict of ids->names? – Roy Iacob May 27 '14 at 00:43
billiard is definitely the key, but this solution does not get the name, as requested. I added a small update in my answer. – cacois Aug 19 '14 at 12:57

score 7 · Answer 3 · answered Sep 15 '16 at 12:25

7

I also needed the worker name for reporting purposes so I tried @cacois solution but it doesn't seem to work with eventlet (current_process() doesn't have an initargs attribute). So I'll leave my solution here for future references:

from celery import task

@task(bind=True)
def getName(self):
    return self.request.hostname

The name of the attribute sounds weird to me, but that contains the name specified with the "-n" option when launching the worker. self works like you would expect from a class method, you don't need to specify it when calling the function (eg: getName.delay()).

answered Sep 15 '16 at 12:25

Adrian Macias

95
1
1

You don't need to do this every time you run a task. Way easier to capture the worker name on connect. See my answer bellow. – Teo Sibileau Jul 25 '17 at 16:57
Hostname is the name of the server, not the name of the worker. – Cerin Mar 18 '21 at 23:16

cacois · Answer 4 · 2014-08-19T16:45:34.973

3

You were originally looking for the name you put in with the -n flag, right? It's in the initargs array. Here's a modified version of the answer that gets you that:

from celery import task
from billiard import current_process

@task
def getName():
    p = current_process()
    return p.initargs[1].split('@')[1]

edited Aug 19 '14 at 16:45

answered Aug 19 '14 at 12:56

cacois

2,036
2
19
19

Unfortunately this seems to no longer work with recent celery versions. `current_process()` now returns a `ForkProcess` object, which has no `initargs`. – ForeverWintr Oct 04 '18 at 19:29

Get the name of celery worker from inside a celery task?

4 Answers4