2

How can I implement resolution of conflicts when I use Tarantool in multi master scenario?

I'm developing a service which should be highly available, so decided to use nginx as a load balancer (with backup directive) for two nodes of tarantool (with disabled read only option). It retries failed requests to other node, but in case of network issues (for instance, between nodes of tarantool) conflicts may occur.

How can I implement one of the following scenarios:

  1. Choose a newer tuple on each node
  2. Custom logic (may be another space for conflics and so on)

Another question is how can I define unique nullable compound index (null is a value which can occur multiple times)

| id | user_id | type | {some data} |

Indexes:

id - PK
user_id + type - unique nullable tree index (type is nullable)
user_id has non unique tree index
DarkWiiPlayer
  • 6,871
  • 3
  • 23
  • 38
Nick Karlov
  • 51
  • 5
  • So you want `user_id + type` to be unique, except when `type` is null? – DarkWiiPlayer Jun 24 '19 at 13:03
  • @DarkWiiPlayer, very close. I asked the question here: https://stackoverflow.com/questions/56737313/nullable-unique-compound-index-in-tarantool – Nick Karlov Jun 24 '19 at 13:11
  • That would be difficult to achieve; if possible, you should consider using a master-slave setup instead or just build your database interactions in a conflict-free way. – DarkWiiPlayer Jun 24 '19 at 13:36

3 Answers3

3

1) You need a before_replace trigger on the space which may have conflict to implement the rules of conflict resolution of your application.

https://www.tarantool.io/en/doc/2.1/book/box/box_space/#box-space-before-replace

In the trigger you can compare old and new replica record and choose which one to use (or skip the update entirely, or merge two records together).

2) You need to set the trigger at the right time, before the space starts to receive any updates. The way you usually set the before_replace trigger is right when the space is created, so you need a trigger set another trigger on system space _space, to capture the moment when your space is created and set the trigger there. This can be on_replace trigger, https://www.tarantool.io/en/doc/2.1/book/box/box_space/#box-space-on-replace the difference between before_replace and on_replace is that *on_replace is called after a row is inserted into the space, and before_replace is called before. 3) To set _space:on_replace() trigger you also need the right timing. The best timing to use is when _space is just created, which is box.ctl.on_schema_init() trigger. https://www.tarantool.io/en/doc/2.1/book/box/box_ctl/#lua-function.box.ctl.on_schema_init

Kostja
  • 1,607
  • 10
  • 17
3

Regarding second point from Kostja's answer (combination of on_ctl_init+_space:on_replace), there is one more trick: you'll need to utilize box.on_commit to get acces to the space being created. Resulting snippet would be the following:

local my_space_name = 'ny_space'
local my_trigger = function(old, new) ... end
box.schema.on_schema_init(function()
    box.space._space:on_replace(function(_, new_space)
        if new_space.name == my_space_name then
            box.on_commit(function()
                box.space[my_space_name]:before_replace(my_trigger)
            end
        end
    end)
end)
Dmitry Sharonov
  • 471
  • 2
  • 12
0

Regarding 2) I hit the problem. Enabling trigger "right when the space is created" in my case cause read_only errors. The reason is: trigger tries to upsert to statistic table while reading from WAL.

local function before_replace(old, new)
    -- collision resolving here
    if box.session.type() ~= 'applier' then
        box.space.stat:upsert(
            { "key", 0 },
            {
                {"+", stat.COUNT, 1}
            })
    end
    return
end

In that case i need to enable trigger only then WAL had been read. And before replication sync starts(otherwise i can hit collision, or loose statistic).I found this is the right time to do it. I enable a trigger after box.info.status changed from "loading". Like this:

local my_space_name = 'myspace'
local function loading_before_replace(old, new)
    if box.info.status == "loading" then
        return
    end
    box.space.my_space_name:before_replace(before_replace, loading_before_replace)
    return before_replace(old,new)
end

local function _space_on_replace(old, new)  
    if not new or not new.name then
        return
    end
    -- skip system spaces
    if string.startswith(new.name, "_") then
        return
    end

    if new.name == my_space_name  then
        box.on_commit(function()
            box.space.my_space_name:before_replace(loading_before_replace)
        end)
    end
end

local function set_triggers()
    box.ctl.on_schema_init(function()
        box.space._space:on_replace(_space_on_replace)
    end)
end

So before_replace() trigger will be executed and enabled on the first commitment to myspace after initial WAL reading.

Maybe it's possible to hit trigger on box.info.status changed? This could make code cleaner. But i don't know if it's possible.

Vasil
  • 45
  • 5