Restore lost tasks Redis python

Question

I was thinking about the way to secure accomplishment of all tasks stored in Redis queues in case of server shutdown e.g. My initial thought was to create an instance of job-description and saving it to database. Something like:

class JobDescription(db.Model):
    id = Column(...)
    name = Column(...)
    queue_name = (...)
    is_started = (Boolean)
    is_finished = (Boolean)
    ....

And then updating boolean flags when necessary.

So on Flask/Django/FastApi application startup I would search for jobs that are either not started/not finished.

My question is - are there any best practices for what I described here or better ways to restore lost jobs rather than saving to db job-descriptions?

score 1 · Accepted Answer · answered Jul 05 '21 at 00:09

You're on the right track. To restore Rq tasks after Redis has restarted, you have to record them so that you can determine what jobs need to be re-delayed (queued).

The JobDescription approach you're using can work fine, with the caveat that as time passes and the underlying table for JobDescription gets larger, it'll take longer to query that table unless you build a secondary index on is_finished.

You might find that using datetimes instead of booleans gives you more options. E.g.

started_at = db.Column(db.DateTime)
finished_at = db.Column(db.DateTime, index=True)

lets you answer several useful questions, such as whether there are interesting patterns in job requests over time, how completion latency varies over time, etc.

Restore lost tasks Redis python

1 Answers1