Rethinkdb - Atomic retrieve/update row with specific condition

Question

I have a bunch of celery workers that need to get single record(row) from the database.

It should only be returned if 60 seconds passed since "last_used" field and as soon as returned, it should update the "last_used" with current time(so other workers that have unlimited retries with 0 delay don't get the same one)

Is it possible to do all of this at db level? I wouldn't have any other source updating this "last_used" field.

Here is how single row looks like.

{"id": "..." "last_used": "2016-06-31 00:37:21.241833", "item": "string"}

I tried:

conn = r.connect(host='localhost', port=28015)
do = r.db('database').table('tb').order_by(index=("last_used")).limit(1).update(
{'last_used': r.now()}
, return_changes=True).run(conn)

And it doesn't work, multiple workers get returned the same row before it is changed.

score 0 · Answer 1 · edited May 23 '17 at 12:14

I think I did it, tried running 50 workers against it with 0 conflicts. But I am still a newbie with rethinkdb and would love to hear your thoughts on this, borrowed from here.

@app.task(bind=True, default_retry_delay=0, max_retries=999)
def ss(self):        
    conn = r.connect(host='localhost', port=28015)
    do = r.db('').table('').order_by("last_used").filter(
    r.now() - r.row['last_used'] > 10
    ).filter({'status': 1}).limit(1).update(
    r.branch(r.row["status"] == 1, {'status': 2, "last_used": r.now()}, {}),
    return_changes=True).run(conn)


    try:
        got_item = do['changes'][0]['new_val']['id']
        last_used = do['changes'][0]['new_val']['last_used']
    except:
        # print('error', do)
        raise self.retry()

    if got_item:
        # Do stuff that required unique row here....
        print(got_item,'\n',last_used)
        time.sleep(random.randrange(1,5))
        r.db('').table('').get(got_item).update({"status": 1}).run(conn)

#start workers
for i in range(50):
    ss.delay()

I hope with this, workers are guaranteed to work with unique item inside "if got_item" block.

Rethinkdb - Atomic retrieve/update row with specific condition

1 Answers1