Celery task in Flask is not finishing

Question

I am writing a Flask server with a long calculation "optimal_portfolio()". I want the client to send a request to start the calculation in the background and another later request to get the result. I tried to use Celery and Redis for the background calculation, but when I try to access the Celery task, I only get task.status = PENDING.

from portfolio_creator.portfolio_creator import optimal_portfolio
from flask import Flask, request, jsonify
from flask_sqlalchemy import SQLAlchemy
from celery import Celery
import os


app = Flask(__name__)

app.config['CELERY_BROKER_URL'] = os.environ['REDISTOGO_URL']
app.config['CELERY_RESULT_BACKEND'] = os.environ['REDISTOGO_URL']

celery = Celery(app.name, broker=app.config['CELERY_BROKER_URL'])
celery.conf.update(app.config)


@celery.task
def calculate_optimal_portfolio(risk_coefficient):
    return optimal_portfolio(risk_coefficient).to_json(orient="records")


@app.route('/portfolio-creator', methods=['POST'])
def set_portfolio():
    risk_coefficient = request.form.get('risk_coefficient', 5)

    task = calculate_optimal_portfolio.apply_async(
        args=[risk_coefficient])

    return jsonify({"task_id": task.id}), 202


@app.route('/portfolio-creator/<task_id>', methods=['GET'])
def get_portfolio(task_id):
    result = celery.AsyncResult(task_id).result

    if result is None:
        return 'Portfolio not ready.', 404

    return result, 200


if __name__ == '__main__':

    app.run()

I also tried using Flask sessions, but that does not seem to work with Celery.

@DaveW.Smith Yes thank you, just fixed it. But that error was made on writing the question, so my initial problem still applies. — Tim Berti, Aug 19 '21 at 06:07
It feels like something that can be fixed with a configuration value. Maybe this could help: https://stackoverflow.com/questions/27357732/celery-task-always-pending — Fnaxiom, Aug 26 '21 at 08:52

score 0 · Answer 1 · answered Jul 20 '23 at 01:09

Had a similar issue where my tasks were being executed but the status was always PENDING.

Turns out you need to configure celery to update its status. I configured my flask app like so and then the status updated correctly.

app.config.from_mapping(
    CELERY=dict(
        broker_url=REDIS_URL,
        result_backend=REDIS_URL,
        task_ignore_result=False,
        task_track_started=True,
    )
)

task_ignore_Result=False was necessary for the SUCCESS status to appear. task_track_started was necessary for the STARTED status to appear.

More info on them here: https://docs.celeryq.dev/en/stable/userguide/configuration.html. If another person reading this doesn't use flask, this answer should be helpful for configuring celery without flask https://stackoverflow.com/a/40831283/5915915

Celery task in Flask is not finishing

1 Answers1