What the error with ConjunctionNode() means?

Question

My application experiences over quota issues and I would like to handle such cases properly in my code. The limit is reached just for Datastore Read Operations, but I get TypeError: ConjunctionNode() requires at least one node exception when try to read the data from the memcache - entries = memcache.get('mykey').

Why this exception happens?

Exception details:

Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 266, in Handle
    result = handler(dict(self._environ), self._StartResponse)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1519, in __call__
    response = self._internal_error(e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1511, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1505, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 1077, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 547, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.3/webapp2.py", line 545, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~myapp/1.373233284460557570/myapp.py", line 595, in get
    entries = memcache.get('mykey')
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/memcache/__init__.py", line 559, in get
    results = rpc.get_result()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 612, in get_result
    return self.__get_result_hook(self)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/memcache/__init__.py", line 624, in __get_hook
    self._do_unpickle)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/memcache/__init__.py", line 271, in _decode_value
    return do_unpickle(value)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/memcache/__init__.py", line 401, in _do_unpickle
    return unpickler.load()
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 550, in __new__
    raise TypeError('ConjunctionNode() requires at least one node.')
TypeError: ConjunctionNode() requires at least one node.

Upd. here is what I save to memcache:

entries = MyModel.query()
entries = entries.fetch(keys_only=True)
entries = random.sample(entries, 10)
entries = [list_key.get() for list_key in entries]
memcache.set('mykey', entries, 60*60*24)

Upd2. Online memcache viewer shows the following value stored (just first part is shown below):

Type: Object ..cgoogle.appengine.ext.ndb.query.Query.q.).q.}q.(U._Query__projectionq.NU._Query__filtersq.cgoogle.appengine.ext.ndb.query.ConjunctionNode.q.).q.}q.U._ConjunctionNode__nodesq.]q.(cgoogle.appengine.ext.ndb.query.FilterNode.q.).q.}q.(U._Filt

It looks like you are storing a Query object in memcache rather than results. See it's trying to unpickle a Boolen expression (ConJunctionNode) used to define a query. I suggest you investigate or show what you are trying to store in "mykey" — Tim Hoffman, Jun 09 '14 at 15:14
@TimHoffman, I've updated the question with what I store there. Does it mean that I use memcache ineffectively? I.e. each time still it reads the database? — LA_, Jun 09 '14 at 15:28
I don't think what your are showing matches with what ndb is trying to unpickle. Can you try an memcache fetch immediately after the set, and see if your able to retrieve things. I would also suggest you dump, or examine using ndb the content of entries just before you store it. Is this on SDK or prod? — Tim Hoffman, Jun 09 '14 at 15:47
This is Prod (since I experience over quota exception, as I've mentioned ;). Due to the same reason I can not dump the data (since I can not read datastore). See updated question - I've added what I see online when display memcache value. — LA_, Jun 09 '14 at 16:17
Interesting moment - since `texts` property in model is repeated, I replace it with one randomly chosen value before storing to the datastore. And after each page refresh it shows the same result (what I wanted to achieve), i.e. it doesn't read datastore with each memcache get. — LA_, Jun 09 '14 at 16:27

score 0 · Answer 1 · answered Sep 25 '14 at 13:57

I believe that this happens simply because you are trying to put a pickled Query object into memcache, but Query objects are not pickle-able.

What I do instead in this case is to convert the Query to a list prior to putting it into the cache. If your code doesn't care about the distinction (that is, if it's not calling .filter() or something like that) then it'll probably work fine for you too.

What the error with ConjunctionNode() means?

1 Answers1