AppEngine achieving strong consistency

Question

I am trying to achieve strong consistency. Let's call my model PVPPlayer:

class PVPPlayer(ndb.Model):
    points = ndb.IntegerProperty()

Every key for the model is created like this:

pvp_player = PVPPlayer(key=ndb.Key(Profile, "test_id", PVPPlayer, "test_id"))

where Profile is parent model:

class Profile(ndb.Model):
    def build_key(cls, some_id):
        return ndb.Key(cls, some_id)

I have 2 REST api url:

1) update_points
2) get_points

In 1) I do :

# I use transaction because I have to update all the models in single batch 
@ndb.transactional(xg=True, retries=3)
def some_func(points):
    pvp_player = ndb.Key(Profile, "test_id", PVPPlayer, "test_id").get()
    pvp_player.points += points 
    pvp_player.put()
    # update other models here`

In 2) I do:

pvp_player = ndb.Key(Profile, "test_id", PVPPlayer, "test_id").get()
return pvp_player.points`

My flow looks like this:

1) update_points()
2) get_points()
3) update_points()
4) get_points()`
...

Problem:

Using get() guarantees strong consistency so what I don't understand is why sometimes as the result of get_points() I get stale data like points were not updated at all.

Example:

POST get_points -> 0
POST sleep 1-3 sec
POST update_points -> 15
POST sleep 1-3 sec
POST get_points -> 15
POST sleep 1-3 sec
POST update_points -> 20
POST sleep 1-3 sec
POST get_points -> 15 !!!`

Is your #2 function also decorated with `@ndb.transactional`? Also, I presume the `update_points` function name in #1 is really `update_points`, right? — Dan Cornilescu, Aug 25 '16 at 15:33
>Is your #2 function also decorated with @ndb.transactional? I tried to add @ndb.transactional for #2 but it doesn't matter - it still gives me stale data from time to time >update_points function name in #1 is really update_points, right? Absolutely — Peter Leontev, Aug 25 '16 at 17:54
You may call some_func "commit" function. It updates PVPPlayer points and does save some other models like PVPMatch (some_func can operate at max 3 entity groups) but i do update PVPPlayer model only once in this request. — Peter Leontev, Aug 25 '16 at 18:02

score 2 · Answer 1 · edited May 23 '17 at 11:46

2

First check your logs, one of the updates must be failing with error, because your logic is basically correct.

Also double-check all your updates are wrapped in transactions to avoid races. Cloud Datastore: ways to avoid race conditions

This case is likely not about consistency issues, but stomping updates, checkout this links for some interesting cases:

http://engineering.khanacademy.org/posts/transaction-safety.htm http://engineering.khanacademy.org/posts/user-write-lock.htm

edited May 23 '17 at 11:46

Community

1
1

answered Aug 26 '16 at 10:30

glmvrml

1,612
2
14
31

I just understood that one of my requests updates my parent model (i.e. Profile) simultaneously with update_points(). That looks like problem – Peter Leontev Aug 26 '16 at 11:29
Do I have to assign different parent key to avoid this problem right? – Peter Leontev Aug 26 '16 at 11:30
Everything that shares the parent key including the parent is in the same entity group. – Tim Hoffman Aug 26 '16 at 12:30
@PeterLeontev wrap all your updates in transactions to avoid race conditions – glmvrml Aug 26 '16 at 13:16

score 1 · Accepted Answer · answered Aug 25 '16 at 15:25

1

Is there a case you exceed the write limit per entity group, that is one update per second? I think this could break the strong consistency of the entity group as mentioned in the documentation.

answered Aug 25 '16 at 15:25

Konstantinos G

46
4

Physical player is the only one who updates PVPPlayer model. The situation when someone does 2 update_points() requests simultaneously for the the same model cannot happen at all according to game logic – Peter Leontev Aug 25 '16 at 17:56
Actually what is even more strange is that sometimes there is 15-20 sec gap between two sequantial update_points() but data are still stale so it behaves exactly like eventual consistency but I cannot understand why – Peter Leontev Aug 25 '16 at 18:00

AppEngine achieving strong consistency

2 Answers2