A a means of viewing task-specific logs via Flower (or some other similar interface)

Question

I have an application/website which I use to run tests. Whenever I run a test, a Celery task is created, and this task goes through the process of running the actual test. The test contacts a 3rd party server, so there are quite a few reasons why the task might fail or hang. This all works fine when everything's run locally; I have direct access to stdout and stderr—they pop up right on the terminal I used to start Celery's worker. If there's an error, a hangup, or any other such thing, I can see it directly, deal with it, and make sure it's handled gracefully in the future.

Eventually, this will be hosted on servers that are independent of my computer, which is where the problem begins:

I'd like a way to access task-specific logs to stdout and stderr (preferably, in real time). I implemented Flower thinking it might do this, but it seems that it doesn't. I've thought about saving the logs to a file, one for each task, and including a "View Log" button link on my site which would allow me to see the logs I'd otherwise see locally—but this is pretty cumbersome. Maybe I could do something like: Generate a link to each running task, and use javascript to update the page in that link w/ the contents of a log file?

I've done some research and haven't found much in the way of this type of logging. Would anyone mind pointing me in the right direction?

score 0 · Answer 1 · answered Jul 24 '20 at 10:15

You can create a LoggingTask object capturing the logs and publishing them as the result of the task (if you're not using the result).

I did not test this, but it should work with a few teaks (you should note it will also eat up exceptions) :

import logging
from io import StringIO
from celery import Task

LOGGER = logging.getLogger(__name__)

class LoggingTask(Task):
    def __call__(self, *args, **kwargs):
        # Only handles the logging capture
        log_stream = StringIO()
        handler = logging.StreamHandler(log_stream)
        try:
            logging.getLogger().addHandler(handler)
            Task.__call__(self, *args, **kwargs)
        catch:
            LOGGER.exception("Error in task")
        finally:
            logging.getLogger().removeHandler(handler)

        handler.flush()

        return log_stream.getvalue()

A a means of viewing task-specific logs via Flower (or some other similar interface)

1 Answers1