I want to delete all flower celery history/logs but it does not work

Question

I do not know how to delete my task related data for flower. I need to clear my Flower data for debugging purposes but I do not know how to do it.

My flower and celery is running on a docker and even when I delete the dockers and their corresponding volumes I still have the data in my flower and celery history data.

My docker-compose configuration :

celery:
    build: ./web
    command: su myuser -c "celery worker --app=project.celery:app --loglevel=info --logfile=log/celery-worker.log --concurrency=4"
    volumes:
      - ./web:/usr/src/app
    env_file: .env
    environment:
      DEBUG: 'true'
    links:
      - postgres
      - redis

  flower:
    build: ./web
    expose:
      - '5555'
    ports:
      - '5555'
    command: su myuser -c "celery flower --app=project.celery:app --url_prefix=flower --persistent=True"
    volumes:
      - ./web:/usr/src/app
    env_file: .env
    environment:
      DEBUG: 'true'
    links:
      - postgres
      - redis

My DockerFile :

FROM python:3.5-stretch

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

# pip install
COPY requirements.txt /usr/src/app/
RUN pip install --no-cache-dir -r requirements.txt

# create unprivileged user (celery)
RUN adduser --disabled-password --gecos '' myuser

Answer 1

It's not a Docker problem but a Flower related problem : If the flower command is started with the option --persistent=True then the data from celery is stored in a python shelve (persistent dictionary, https://docs.python.org/3/library/shelve.html). This data is stored on disk at the root of the folder from which we start the flower docker. By default the file is named ´flower´.

The place where this file is stored can be changed by appending the option --db=data/flower.

To delete the history you can simply delete the file containing this data.

Answer 2

I implemented a new API for deleting tasks

Events:
  None delete_all_tasks()

Example: delete tasks

import celery
import os
import datetime

broker = os.environ.get('CELERY_BROKER_URL')
app = celery.Celery('tasks', broker=broker)
flower = Flower(capp=app, options=flower_options)


flower.events.delete_all_tasks()

Then we can a celery beat scheduler, that runs each hour, and delete the tasks

@app.task(queue='cleanup_tasks')
def clean_up_tasks():
    from flower.app import Flower

    flower_options = object()
    flower_options.db = 'flower'
    flower_options.persistent = True
    flower_options.purge_offline_workers = 1

    flower = Flower(capp=app, options=flower_options)
    flower.events.delete_all_tasks()

@app.on_after_configure.connect
def add_periodic(**kwargs):
    app.add_periodic_task(crontab(hour="*", minute=0), clean_up_tasks.s(), name='cleanup-tasks')

References: https://github.com/mher/flower/issues/1189 https://github.com/mher/flower/pull/1188

Hopefully the PR gets merged soon!

Enjoy <3