2

I am running into an error that I am unable to isolate the root cause of. The error is the following: "ReferenceProperty failed to be resolved: [u'StatusLog', STATUSLOGSID]". This error only occurs sometimes, about once or twice a day. The scripts that generate this error succeed way more often than they fail. The strangest thing about the error is that it is failing to resolve a reference property, which should never be the case (in regards to this situation) because the entities that are being referenced are never deleted by my webapp. Furthermore, I am not generating the keys that are being referenced, the Google App Engine is. The relevant code is listed below.

THE GAE TRANSACTION:

def updateStatus(key):
    hbo = HBO.get(key)
    hbo.updateStatus()
    hbo.put()

class HBOCRON(webapp2.RequestHandler):
    def get(self):
        keys = db.Query(HBO, keys_only = True).filter("inactive = ", False)
        XG_ON = db.create_transaction_options(xg=True)
        for key in keys: db.run_in_transaction_options(XG_ON, updateStatus, key)

app = webapp2.WSGIApplication([('/cron/hbo', HBOCRON)],debug=True)

Two other relevant functions...

def logStatus(self):
    self.status = StatusLog(
        hbo = self,
        prev = self.status,
        date = datetime.datetime.now(),
        on = self.online(),
        up = self.upToDate(),
        dns = self.DNS_update_needed,
        dis = self.manually_disabled).put()

def updateStatus(self):
    status = self.status
    if status is None \
        or status.on != self.online() \
        or status.up != self.upToDate() \
        or status.dns != self.DNS_update_needed:
        self.logStatus()
        self.flagged = True
    elif status.dis != self.manually_disabled:
        self.logStatus()

Traceback:

ReferenceProperty failed to be resolved: [u'StatusLog', 248327L]
Traceback (most recent call last):
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
    return handler.dispatch()
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/python27_runtime/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~hs-hbo/1.357660268729453201/api/hbo/getCheckin.py", line 88, in post
    (hbo, data) = db.run_in_transaction_options(XG_ON, checkinTransaction, self.request)
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/api/datastore.py", line 2476, in RunInTransactionOptions
    ok, result = _DoOneTry(new_connection, function, args, kwargs)
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/api/datastore.py", line 2498, in _DoOneTry
    result = function(*args, **kwargs)
  File "/base/data/home/apps/s~hs-hbo/1.357660268729453201/api/hbo/getCheckin.py", line 33, in checkinTransaction
    hbo.updateStatus()
  File "/base/data/home/apps/s~hs-hbo/1.357660268729453201/shared/datastore.py", line 116, in updateStatus
    return self.logStatus()
  File "/base/data/home/apps/s~hs-hbo/1.357660268729453201/shared/datastore.py", line 102, in logStatus
    prev = self.status,
  File "/base/python27_runtime/python27_lib/versions/1/google/appengine/ext/db/__init__.py", line 3597, in __get__
    reference_id.to_path())
ReferencePropertyResolveError: ReferenceProperty failed to be resolved: [u'StatusLog', 248327L]

Thanks for any insight/help/answers/suggestions!

papezjustin
  • 2,223
  • 6
  • 24
  • 31
  • 1
    Please supply the traceback from the logs and the code fragment where the ReferenceProperty is actually dereferenced. – Guido van Rossum Mar 23 '12 at 03:48
  • I have added the full traceback to my SO post. Thanks for creating such a wonderful scripting language and being active in the Python community. – papezjustin Mar 23 '12 at 16:36

1 Answers1

3

This happens when you attempt to resolve a reference property (by dereferencing it - for instance, (MyModel.MyReferenceProp.foo), and the property being referenced no longer exists - because it has been deleted.

You need to modify your code to catch this exception when you dereference an entity that may have been deleted, and handle it appropriately.

Nick Johnson
  • 100,655
  • 16
  • 128
  • 198
  • Hi Nick, thanks for responding to my post. I understand that this happens when you attempt to resolve a reference property; however, my coworkers and I never delete any StatusLog entities whatsoever. So we are curious as to how it is failing to resolve a reference property when the entity that is being deleted is never modified or deleted. – papezjustin Mar 23 '12 at 16:40
  • 1
    @JustinPapez In that case, you should catch the exception, and examine the entity in question to see what's wrong. – Nick Johnson Mar 24 '12 at 07:58
  • Yes, that is what we are currently doing; however it just seems kind of "hackish". It also occurs on different entities of the same class. Thanks for your help. – papezjustin Mar 24 '12 at 19:35
  • @JustinPapez Well, it's debugging. Debugging is often hackish. What did you learn? – Nick Johnson Mar 25 '12 at 09:09
  • Unfortunately nothing yet. Even after further inspection of the code and further debugging the problem is still occurring every now and then. – papezjustin Mar 26 '12 at 16:08
  • @JustinPapez What did you learn about the entities for which it occurs on, though? What are their reference properties set to? What about the entities they point to? You wouldn't be getting this error if the entity being referenced was present and accounted for. – Nick Johnson Mar 26 '12 at 17:47
  • Sorry, it seems like I didn't understand your question correctly the first time. We have a class called HBO that references a StatusLog entity of the StatusLog class. Each StatusLog entity references its previous StatusLog entity. A StatusLog entity has 7 properties: date (DateTimeProperty), hbo (ReferenceProperty to HBO entity), prev (SelfReferenceProperty that references the HBO entity's previous StatusLog entity), and then four boolean properties that are irrelevant (online, uptodate, dnsupdate, and manually disabled). StatusLog entities are never deleted or manipulated. – papezjustin Mar 26 '12 at 20:25
  • Only new ones are created and when they are created the key is not manually created, it is automatically created by App Engine. – papezjustin Mar 26 '12 at 20:25
  • @JustinPapez Clearly, though, there are entities in your datastore which reference other entities that do not exist. Again, what did you learn by examining the specific entities for which the exception occurs? – Nick Johnson Mar 27 '12 at 09:29