How to run non transaction pipeline from a client.Watch function in go-redis

Question

I have a scenario similar to the following :

txf := func(tx *redis.Tx) error {
  // Phase 1:  
  // read some stuff by several get/hget request which I want to send with pipeline
  // to avoid unnecessarily rounds to the redis server 

  // Phase 2: Prepare new data based on read data

  // Phase 3: Write the new data with a transaction to use the watch protection if original keys changed
  _, err = tx.Pipelined(func(pipe redis.Pipeliner) error {
             // pipe handles the error case
             pipe.Set(key, value, 0)
             return nil})
  return err
}
err := client.Watch(txf, key)

From reading the library code it seems that tx.TxPipeline and tx.Pipeline both returns in this case a multi/exec implementation which means that if I'll use it for reading data (phase 1) I'll lose the watch protection in the second pipeline I use to update the data.
Any solutions will be appriciated.

Thanks to LeoMurillo, this is now fixed in the https://github.com/go-redis/redis/pull/1234 — Avner Levy, Feb 01 '20 at 04:38

score 1 · Accepted Answer · answered Jan 10 '20 at 11:19

You can use a pipeline using client.Pipelined(... instead of tx..Pipelined(..., but it will be sent to the redis server using another connection from the go-redis pool (another client from the redis server perspective). I don't think that's an issue though.

go-redis transactions use a sticky connection to ensure the whole transaction, starting with the WATCH, is sent from the same connection. The internal tx.baseClient is unexported. There is no way to send a pipeline using the same connection.

txf := func(tx *redis.Tx) error {
  // Phase 1:  
  var getPipe *redis.StringCmd
  cmds, err := client.Pipelined(func(pipe redis.Pipeliner) error {
        getPipe = pipe.Get("getPipe")
        pipe.Set("pipe1", "p1", 0)
        return nil
  })
  fmt.Println(getPipe)
  fmt.Println(cmds)
  val, _ := getPipe.Result()
  fmt.Println("Value read for 'getPipe':", val)

  // Phase 2: Prepare new data based on read data

  // Phase 3
  _, err = tx.Pipelined(func(pipe redis.Pipeliner) error {
    // pipe handles the error case
    pipe.Set(key, value, 0)
    return nil})
  return err
}
err := client.Watch(txf, key)
fmt.Println(client.Get(key), err)

The output of the above go program:

get getPipe: preVal
[get getPipe: preVal set pipe1 p1: OK]
Value read for 'getPipe': preVal
get myKey: insideMulti <nil>

And this is what I see with the MONITOR command on redis-cli:

1 ...1:65506] "watch" "myKey"
2 ...1:65507] "get" "getPipe"
3 ...1:65507] "set" "pipe1" "p1"
4 ...1:65506] "MULTI"
5 ...1:65506] "set" "myKey" "insideMulti"
6 ...1:65506] "EXEC"
7 ...1:65506] "unwatch"
8 ...1:65506] "get" "myKey"

Note lines 2 & 3 are on a different port

Thanks! That was what I used for a workaround, but I wondered why does Pipeline has a different semantic compared to client.Pipeline. Why can't you pack a bunch of read-only operations on a Tx connection and send them for faster execution? — Avner Levy, Jan 10 '20 at 15:10
When Tx MultiExec was renamed to Pipelined (years ago), the regular non-tx Pipelined was made unavailable. I just commented on the issue you submitted :-) https://github.com/go-redis/redis/issues/1233 — LeoMurillo, Jan 10 '20 at 16:14

How to run non transaction pipeline from a client.Watch function in go-redis

1 Answers1