Celery tasks uncaught exceptions not being sent to Sentry

Question

I'm currently working with Celery tasks in a Django based project. We have raven configured to send all uncaught exceptions and log messages to Sentry, as described in the documentation.

Everything works pretty good, except for uncaught exceptions inside celery tasks. For example, if I run this task:

@app.task
def test_logging():
    log.error('Testing logging inside a task')
    raise IndexError('Testing exception inside a task')

I only see in Sentry the log.error(...) but not the IndexError uncaught exception. I've tried using a try-except block around the exception with a log.exception(...) inside and it did work, but I think it's not scalable to catch all exceptions like this.

So, the problem are only uncaught exceptions that somehow are not handled properly.

These are my current package versions:

celery (3.1.17)
raven (5.1.1)
Django (1.7.1)

Would you help me to move in some direction?

Thanks for your time!

have you already tried http://raven.readthedocs.org/en/latest/integrations/celery.html and http://docs.celeryproject.org/en/latest/configuration.html#celeryd-hijack-root-logger ? — DRC, Dec 18 '14 at 17:46
Thanks DRC, finally the solution using `register_signal(client)` worked, and now we see uncaught exceptions logged correctly in Sentry. Cheers! — Martin Zugnoni, Dec 23 '14 at 12:31

score 17 · Accepted Answer · edited Aug 09 '16 at 09:19

17

As described by DRC in the comment up there, we finally got to the solution using this approach: https://docs.getsentry.com/hosted/clients/python/integrations/celery/

Basically doing this:

import celery

class Celery(celery.Celery):

    def on_configure(self):
        if hasattr(settings, 'RAVEN_CONFIG') and settings.RAVEN_CONFIG['dsn']:
            import raven
            from raven.contrib.celery import (register_signal,
                                              register_logger_signal)

            client = raven.Client(settings.RAVEN_CONFIG['dsn'])
            register_logger_signal(client)
            register_signal(client)


app = Celery('myapp')
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

Thanks for your time.

edited Aug 09 '16 at 09:19

eugene

39,839
68
255
489

answered Dec 23 '14 at 12:34

Martin Zugnoni

1,439
2
14
21

In which file do you write this configuration for celery to read it? – jperelli Dec 17 '15 at 13:16
1

I have a dedicated `celery.py` module where all the celery configuration is located. Then that configuration is imported in other module `tasks.py` where I have all the tasks defined. – Martin Zugnoni Dec 18 '15 at 01:29
3

What's the alternative for their new `sentry-sdk` API? – Nikita Shupeyko Jul 17 '19 at 11:23

score 9 · Answer 2 · answered Mar 31 '20 at 09:19

9

The chosen answer is correct as the question states raven is being used. However raven has been deprecated in favour of the sentry SDK.

To configure django and celery with the sentry SDK:

First add sentry-sdk to your requirements file.

Then, in your django setttings file:

import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration
from sentry_sdk.integrations.celery import CeleryIntegration

sentry_sdk.init(
    dsn="your_sentry_DSN",
    integrations=[DjangoIntegration(), CeleryIntegration()],
    send_default_pii=True
)

Sources:

Get your DSN: https://docs.sentry.io/clients/java/config/#setting-the-dsn
Sentry Django integration: https://docs.sentry.io/platforms/python/django/
Sentry Celery integration: https://docs.sentry.io/platforms/python/celery/

answered Mar 31 '20 at 09:19

gdvalderrama

713
1
17
26

Hi, I have added these changes to my code but it is not capturing celery task errors. Is any other changes are needed in the codebase. Can you please help?@guival – Anjitha Jul 15 '21 at 05:04
@Anjitha No, that should be it. Is it correctly capturing other errors? – gdvalderrama Jul 15 '21 at 09:51
Yes, It's tracking all errors that are not related to celery. Is there any additional settings in the celery configuration file? – Anjitha Jul 16 '21 at 10:10
@Anjitha Can't think of anything right now, maybe you should open your own question. – gdvalderrama Jul 19 '21 at 08:20
@gdvalderrama I propose making a new question for `sentry-sdk` (and [self-answering](https://stackoverflow.com/help/self-answer) with your answer here). The chosen answer was framed in the context of the legacy raven package and has an answer catered to that. They're different packages. – tony Jan 26 '22 at 19:43

Celery tasks uncaught exceptions not being sent to Sentry

2 Answers2