redis get and set key expiry with transaction

Question

I am using etaty redisscala (https://github.com/etaty/rediscala) client. Here is my function

private def getVersionTime(db: RedisClient, interval: Long)(implicit ec: ExecutionContext): Future[Long] = {

import akka.util.ByteString
import redis.ByteStringFormatter

implicit val byteStringLongFormatter = new ByteStringFormatter[Long] {
  def serialize(data: Long): ByteString = ByteString(data.toString.getBytes)
  def deserialize(bs: ByteString): Long = bs.utf8String.toLong
}

db.get[Long]("versionTime").map {
  case Some(v) => loggerF.info(s"Retrieved version time ${v}")
    v
  case None => val current = System.currentTimeMillis()
    db.setex[Long]("versionTime", (current / 1000) + interval, current)
    loggerF.info(s"set version time ${current}")
    current
}

}

Here is my test. This test calls above method

it("check with multiple tasks"){
  val target = 10
  val latch = new java.util.concurrent.CountDownLatch(target)
  (1 to target).map{t =>
    getVersionTime(prodDb, 10).map{r => print("\n" + r); latch.countDown()}
  }
  assert(latch.await(10, TimeUnit.SECONDS))
}

Output of test

14:52:46.692 [pool-1-thread-12] INFO EndToEndITTests - set version time 1548062566687 14:52:46.693 [pool-1-thread-6] INFO EndToEndITTests - set version time 1548062566687 14:52:46.693 [pool-1-thread-20] INFO EndToEndITTests - set version time 1548062566687 14:52:46.692 [pool-1-thread-2] INFO EndToEndITTests - set version time 1548062566686 14:52:46.692 [pool-1-thread-10] INFO EndToEndITTests - set version time 1548062566687 14:52:46.693 [pool-1-thread-8] INFO EndToEndITTests - set version time 1548062566687 14:52:46.692 [pool-1-thread-4] INFO EndToEndITTests - set version time 1548062566686 14:52:46.692 [pool-1-thread-11] INFO EndToEndITTests - set version time 1548062566687 14:52:46.692 [pool-1-thread-9] INFO EndToEndITTests - set version time 1548062566687 14:52:46.692 [pool-1-thread-7] INFO EndToEndITTests - set version time 1548062566687

Expected behaviour is - set version time should come once and for rest of the threads Retrieved version time should be printed. I think I need to use transaction here so that get and setex wrapped inside watch and exec

  private def getVersionTimeTrans(db: RedisClient, interval: Long): Long = {
    import akka.util.ByteString
    import redis.ByteStringFormatter

    implicit val byteStringLongFormatter = new ByteStringFormatter[Long] {
      def serialize(data: Long): ByteString = ByteString(data.toString.getBytes)
      def deserialize(bs: ByteString): Long = bs.utf8String.toLong
    }

    val redisTransaction = db.transaction()
    redisTransaction.watch("versionTime")
    val result: Future[Long] = redisTransaction.get[Long]("versionTime").map {
      case Some(v) => loggerF.info(s"Retrieved version time ${v}")
        v
      case None => val current = System.currentTimeMillis()
        redisTransaction.setex[Long]("versionTime", (current / 1000) + interval, current)
        loggerF.info(s"set version time ${current}")
        current
    }
    redisTransaction.exec()
    val r = for {
      i <- result
    } yield {
      i
    }
    Await.result(r, 10 seconds)
  }

test

it("check with multiple threads "){
  val target = 10
  val latch = new java.util.concurrent.CountDownLatch(target)
  (1 to target).map{t =>
    Future(getVersionTimeTrans(prodDb, 10)).map{r => latch.countDown()}
  }
  assert(latch.await(10, TimeUnit.SECONDS))
}

For this test too, output is same. I couldn't figure it how to wrap it inside transaction properly. Please help.

SergGr · Answer 1 · 2019-01-22T02:07:08.003

Looking into the rediscala implementation it seems that you can't use optimistic locking that I suggested in the original answer (see below) because in rediscala the TransactionBuilder will not send the WATCH command until the EXEC command which makes it pretty useless. There is an old closed bug on GitHub which refers to another SO question with exactly this scenario and the answer is

In rediscala, you can't read inside a transaction, because you will be blocking the client for other requests. I suggest you try to look if you can do your check in a LUA script. (transform the transaction in a lua script)

and a few months later

It is closed because not advisable to implement
You should use http://redis.io/commands#scripting

It seems this is in the same state since the 2014 and I don't think it will ever be changed.

Original answer

I don't have alive Redis to test it, but looking into the Redis transaction docs it looks Redis do not support SQL-style transactions as it seems you imagine. It supports atomic operations but you can't do "start transaction-get data-check-possibly modify-commit" cycle. All the "get data" commands will be queued before finally an EXEC command arrives. It means you can't do any checks between get and set inside the same transaction.

If you look at the "Optimistic locking using check-and-set" section of that doc, you can see that the correct way to implement your behavior is:

Create a transaction with watch for the key
Get the value outside of the transaction
Check the value and if update is required, issue an update inside the transaction. (Don't forget to UNWATCH in case if no updates are required)
Execute the transaction
Check if the transaction failed, if so repeat the whole cycle. If not - you are the winner who written the value.

score 0 · Accepted Answer · answered Mar 04 '19 at 15:41

0

I solved the problem using LUA script

answered Mar 04 '19 at 15:41

Abhay

928
1
10
28

redis get and set key expiry with transaction

2 Answers2