How to use redis lock to prevent asynchronous Job in Python

Question

Currently I am using Redis to perform the QueueJob(Priority Queue) in Python code(Odoo) and I meet the asynchronous problem (There will have some jobs doing the same task - update/delete in the same record at the same time).

As I read from Redis documentation that I need to implement a LOCK to prevent the the asynchronous problem. However, I don't why the lock does not run as I expect.

Below is my code:

import redis
import redis_lock #(python-redis-lock lib)

class PriorityQueue(object):
    def __init__(self, queue_name):
        ...
        self.redis = redis.StrictRedis(...)
        self.redis_lock = redis_lock.Lock(self.redis, queue_name)


    def first(self):
        if self.redis_lock.acquire(blocking=False):
            print("Perform task")
            job = self.redis.zrevrange(self.queue_name, 0, 0)[0]
            job_data = json.loads(job.decode("utf-8"))
            return ChannelJob(job_data)
        else:
            print("Lock is used by other job")

    def pop(self):
        job = self.redis.zpopmax(self.queue_name, count=1)
        job_data = json.loads(job.decode("utf-8"))
        return ChannelJob(job_data)

The redis_lock.acquire() is always return False, Please help me to solve this problem.

score 1 · Answer 1 · answered Feb 21 '20 at 06:19

You forget to release the lock.

The redis_lock.acquire() is always return False

Actually thats not true. For the first time at least a single job acquire the lock. And as you forget to release the lock the lock persists. And any subsequent request failed. To solve this problem you need to release the locks self.redis_lock.release(). Here is an example:

def first(self):
        if self.redis_lock.acquire(blocking=False):
            print("Perform task")
            job = self.redis.zrevrange(self.queue_name, 0, 0)[0]
            job_data = json.loads(job.decode("utf-8"))

            // do the job ; you can also release the lock here.
            job_result = ChannelJob(job_data)
            // release the lock, so other can acquire it.
            self.redis_lock.release()
            return job_result 
        else:
            print("Lock is used by other job")

To learn more about the locks: Please visit this links

Also, if there is no timeout defined for the lock, manually find that redis key and delete it. — MD Ruhul Amin, Feb 21 '20 at 06:21

How to use redis lock to prevent asynchronous Job in Python

1 Answers1