Golang - lock per value

Question

I am writing a golang api that accepts a tableName value and a updEpoch value, ie:

curl -F "tableName=abc" -F "updEpoch=123" myhost:8080/singleroute
curl -F "tableName=abc" -F "updEpoch=456" myhost:8080/singleroute
curl -F "tableName=def" -F "updEpoch=123" myhost:8080/singleroute
curl -F "tableName=def" -F "updEpoch=345" myhost:8080/singleroute

I want to allow multiple different tableName requests to be handled in parallel BUT only 1 request per tableName at the same time. So in above example, if above 4 requests fired at same time, then 1st and 3rd should be able to run at same time (as unique tableNames), but 2nd will only start once 1st finishes and 4th will only start once 3rd finishes. When I was researching mutex no example seemed to fit this case, I don't want to hardcode abc/def.etc anywhere in the code as same rule should apply to any arbitrary tableName.

my guess based on Crowman's help:

package main

import (
    "fmt"
    "sync"
    "time"
    "http"
)
km := KeyedMutex{}
type KeyedMutex struct {
    mutexes sync.Map // Zero value is empty and ready for use
}

func (m *KeyedMutex) Lock(key string) func() {
    value, _ := m.mutexes.LoadOrStore(key, &sync.Mutex{})
    mtx := value.(*sync.Mutex)
    mtx.Lock()

    return func() { mtx.Unlock() }
}

func myFunc(key string, data string) string {
  //do some stuff
  return "done for key:"+key+", data: "+data
}

func main() {
    key := //some form value sent to my api 
    data := //some form value sent to my api 
    unlock := km.Lock(key)
    defer unlock()
    retVal := myFunc(key, data)
}

Does this answer your question? [Can I lock using specific values in Go?](https://stackoverflow.com/questions/60198582/can-i-lock-using-specific-values-in-go) — Peter, Oct 28 '20 at 06:19
@Peter: Per the singleflight documentation "if a duplicate comes in, the duplicate caller waits for the original to complete and *receives the same results*" - the OP here does not want the highlighted behavior, so singleflight seems unsuitable. — Crowman, Oct 30 '20 at 01:13

score 19 · Answer 1 · answered Oct 30 '20 at 16:52

19

You can use a sync.Map with your table name as a key and a *sync.Mutex as the value.

For example:

package main

import (
    "fmt"
    "sync"
    "time"
)

type KeyedMutex struct {
    mutexes sync.Map // Zero value is empty and ready for use
}

func (m *KeyedMutex) Lock(key string) func() {
    value, _ := m.mutexes.LoadOrStore(key, &sync.Mutex{})
    mtx := value.(*sync.Mutex)
    mtx.Lock()

    return func() { mtx.Unlock() }
}

func main() {
    wg := sync.WaitGroup{}
    km := KeyedMutex{}

    for _, job := range []struct {
        key  string
        data string
    }{
        {key: "abc", data: "123"},
        {key: "abc", data: "456"},
        {key: "def", data: "123"},
        {key: "def", data: "456"},
    } {
        var job = job
        wg.Add(1)

        go func() {
            defer wg.Done()

            unlock := km.Lock(job.key)
            defer unlock()

            fmt.Printf("%s:%s mutex acquired\n", job.key, job.data)
            time.Sleep(time.Second * 1) // To ensure some goroutines visibly block
            fmt.Printf("%s:%s done\n", job.key, job.data)
        }()
    }

    wg.Wait()
}

with sample output:

crow@mac:$ ./mut
def:456 mutex acquired
abc:456 mutex acquired
abc:456 done
def:456 done
abc:123 mutex acquired
def:123 mutex acquired
def:123 done
abc:123 done
crow@mac:$

to show that requests with distinct table names acquire a mutex immediately, but requests with the same table name are serialized.

answered Oct 30 '20 at 16:52

Crowman

25,242
5
48
56

would you be able to remove the sample data, test parts of the code? ie just leaving the core functionality – tooptoop4 Oct 30 '20 at 17:55
@tooptoop4: No, I wouldn't, as that would reduce the quality of the answer. You can download the code and do that, however. – Crowman Oct 30 '20 at 18:05
is wg := sync.WaitGroup{} part of the solution or test? – tooptoop4 Oct 30 '20 at 19:48
@tooptoop4: All the code is part of the answer. The `WaitGroup` is used to ensure the program doesn't exit until all the goroutines have completed. – Crowman Oct 30 '20 at 22:30
I put some code in the Q, does it look right? My trouble with your answer is that the usable code is mixed in with the test. I guess KeyedMutex should be a global variable not initialised on each api call – tooptoop4 Oct 31 '20 at 20:58
The mutexes are never removed from the map? – stoffen Mar 09 '22 at 19:47
@stoffen: Correct. – Crowman Mar 09 '22 at 20:41
Is it make sense to change the order of type casting and .Lock() to keep lock closer to loading from map? Like this: value.(*sync.Mutex).Lock(); mtx := value.(*sync.Mutex) – Oleg Neumyvakin Dec 27 '22 at 14:48
Any tips to support removal? – fields Jan 27 '23 at 18:23

score 0 · Answer 2 · edited Mar 20 '23 at 16:40

0

You can create a package level var map[string]*sync.Mutex and lock corresponding mutex that you will get/create by table name.

edited Mar 20 '23 at 16:40

blackgreen

34,072
23
111
129

answered Oct 28 '20 at 00:34

Alexander Trakhimenok

6,019
2
27
52

2

Keep in mind the concurrency rules of maps still forbid concurrent read, or read and write. So you should also use a [`sync.RWMutex`](https://golang.org/pkg/sync/#RWMutex) to lock the entire map if you need to assign new keys at runtime – Hymns For Disco Oct 28 '20 at 01:29
2

or just use sync.Map – Liu Wenzhe Oct 28 '20 at 04:11
3

@tooptoop4 do you have a non working example that demonstrates you tried and with a comment that highlights issue you are facing? – Alexander Trakhimenok Oct 28 '20 at 19:50
@tooptoop4 your code looks almost good to me except you have to instantiate `km := KeyedMutex{}` outside of your worker function (main in the example). Probably should be a package level variable or store it in way that it is accessible to all requests. Also you can do small optimization and do `return mtx.Unlock` instead of `return func() { mtx.Unlock() }` though maybe compiler will do that automatically. – Alexander Trakhimenok Nov 03 '20 at 14:39

Golang - lock per value

2 Answers2