1

I'm modelling data, which IDs should be with auto increment. Actually, I've made working model, but need some advise from datastore guru.

There is my code for ID generation:

class AutoIncrementModel(ndb.Model):
  entity_id = ndb.IntegerProperty('eID')
  def _pre_put_hook(self):
    if self.key and self.key.id(): return
    latest = self.__class__.query().order(-self.__class__.entity_id).get()
    self.entity_id = latest and latest.entity_id + 1 or 1
    while self.__class__.get_by_id(self.entity_id): self.entity_id += 1
    self.key = ndb.Key(self.__class__.__name__, self.entity_id, parent=self.key and self.key.parent() or None)
    self.put()

This code generates straightforward IDs, but I'm quite not sure is it good enough.

UPD: This code fails. Several entities can be written with same key and data can be overwritten.

№1. Can it cause problem with data loss? "While loop" preserves app from generating ID. But I'm not sure there is no possibility for data to be overwritten.

№2. May transaction like this make saving better?

  def _pre_put_hook(self):
    def callback():
      while self.__class__.get_by_id(self.entity_id): self.entity_id += 1
      self.key = ndb.Key(self.__class__.__name__, self.entity_id, parent=self.key and self.key.parent() or None)
      self.put()
    if self.key and self.key.id(): return
    latest = self.__class__.query().order(-self.__class__.entity_id).get()
    self.entity_id = latest and latest.entity_id + 1 or 1
    ndb.transaction(callback, xg=True)

UPD: Transaction helps to avoid data loss. This code seems to work much better than first example.

№3. Is there a way to get max ID from group of entities without extra field for index?

TipTop
  • 143
  • 1
  • 2
  • 9
  • 2
    It's difficult to avoid a race condition in creating sequential IDs on a distributed system, and I don't think your first example succeeds in doing so. At the very least you're going to need to involve a transaction (Although I'm not sure whether your second example works either.) (Also see http://stackoverflow.com/questions/3985812/how-to-implement-autoincrement-on-google-appengine although it's possible ndb changes things so I haven't voted to close as duplicate). – Wooble Apr 24 '12 at 11:44
  • 1
    Why do you (think you) need sequential IDs? The datastore comes with a perfectly satisfactory way to generate _unique_ IDs. – Nick Johnson Apr 27 '12 at 04:25

2 Answers2

3

Basically this can't be done. You need an external counter "singleton" and should add sharing for performance should you need more than about one write per second. See How to implement "autoincrement" on Google AppEngine for an discussion of the options.

Community
  • 1
  • 1
max
  • 29,122
  • 12
  • 52
  • 79
0

does not have sense to order by "self.class.entity_id" field beacuse this field is an unique identifier, never has an incremental value, no has sense to add neither.

Harold Sarmiento
  • 127
  • 9