10

Consider the second example in "Uses for Transactions" ("update an entity with a named key, or create it if it doesn't yet exist"):

https://developers.google.com/appengine/docs/java/datastore/transactions

Now consider this scenario. A multiplayer game allows only a single match between any two players. To ensure that, a key is created using each of the player's keys. This key is used as the key of a UniqueMatch entity.

So in order to create a match, a XG transaction is created. Within this transaction:

  1. We check if there isn't a UniqueMatch entity with that key already. If a datastore.get() call using that key does not throw EntityNotFoundException, then we know that a match between those two players already exists, so we rollback() and show an error message to the players.

  2. We put() all entities we need to put in order to create a match. This includes the UniqueMatch entity, plus some other few entities.

  3. The transaction is then committed.

This seems to work fine. However, I noticed I can create two matches between any two players within a short time window. For a small period of time (up to 10-20s in one of the tests, actually), my calls to datastore.get(key) throw EntityNotFoundException even though that key has already been put().

This seems to be eventual consistency. But aren't entity retrivals by key guaranteed to be strongly consistent? Is this guarantee affected by the fact that this is done within a XG transaction?

Thanks in advance,

Dan McGrath
  • 41,220
  • 11
  • 99
  • 130
fnf
  • 414
  • 3
  • 10
  • this may be a similar issue. I got it wrong in the comments, see Guido's comment. http://stackoverflow.com/questions/12367904/write-read-with-high-replication-datastore-ndb/12368444 I was also sure that once you had a key back from a .put you could always retrieve the item but it seems not to be the case. Memcache it perhaps. – Paul Collingwood Oct 08 '12 at 15:15
  • Hmmm I'm not so sure about that, Guido is talking about queries specifically, not entity retrieval by key. For queries, I'd expect eventual consistency (that is properly documented actually). Memcache may be an option, yeah. If I don't find a better way, I'll look into memcache, thanks. – fnf Oct 08 '12 at 15:40
  • sure, you did mention entity by keys so I thought I'd link that discussion. I'l upvote your question. – Paul Collingwood Oct 08 '12 at 17:03
  • This is interesting. Can you drop the cross-group requirement by having step #2 put only the UniqueMatch entity instead, and then see if the same behavior still occurs? Also, concerning the 10-20s delay, is the test being done in the dev server or in deployed app? – Ibrahim Arief Oct 09 '12 at 10:02
  • @IbrahimArief: I'll try to put only the UniqueMatch entity in a non-Xg transaction in step #2 and get back to you. This wouldn't work for production, though, since we would be risking having "dangling" UniqueMatch entities. About the delay, this is for the deployed app. I was unable to reproduce this in the dev server. Thanks. – fnf Oct 09 '12 at 13:13
  • Thanks for following it up. I am currently facing a similar use case as yours. In my case, step #1 and #2 is nearly identical, but I didn't use cross-group option on my transactions since I only need to read/store a single entity. In my local unit test, 10 concurrent threads doing read/store using the same entity key shows the expected strong consistency. However, as you hinted, it might be when I deploy my app I would encounter a similar issue. I'll re-run my test in the deployed app, and let you know how it goes. – Ibrahim Arief Oct 09 '12 at 13:39
  • Some updates. I have deployed and tested my app. From what I see, entity retrievals by key are still strongly consistent within a non-cross-group transaction. Are you still experiencing the issue? – Ibrahim Arief Oct 11 '12 at 16:48

1 Answers1

1

I think the issue you're seeing may be because datastore gets (by key or queries) only see the state of the datastore at the start of the transaction.

From the docs (under isolation and consistency):

In a transaction, all reads reflect the current, consistent state of the Datastore at the time the transaction started. This does not include previous puts and deletes inside the transaction. Queries and gets inside a transaction are guaranteed to see a single, consistent snapshot of the Datastore as of the beginning of the transaction.

Also, outside of the transaction, you'll only see puts that have been committed (docs):

Entities retrieved from the datastore by queries or gets will only see committed data.

A possible solution:

Create the UniqueMatch entity outside of the transaction (appengine won't allow you to put an entity with the same key, so it'll throw an exception if the entity with the same key already exists). You can then create/put the other entities required to create a match inside a transaction (if need be).

Lastly, make sure when creating the key for the UniqueMatch that they key is always created with the players in the same order because key(playerA,playerB)!=key(playerB,playerA)

Rob Curtis
  • 2,245
  • 24
  • 33