How can I cache/memoize my SQLAlchemy functions?

Question

I am using FLask-OAuthlib and want to do some caching/memoization using Flask-Cache. I've got caching setup on my views but I'm having trouble with caching this function:

@oauth.clientgetter
@cache.memoize(timeout=86400)
def load_client(client_id):
  return DBSession.query(Client).filter_by(client_id=client_id).first()

The first time the function is run (not cached yet) it runs fine but when it gets it from cache something gets messed up somehow and says it's an invalid client. I don't know if it's caching it incorrectly or if having the @oauth.clientgetter decorator somehow screws up the caching. Everything works fine without caching and the client is valid. I've tried to move the function around like so, but get the same result:

class Client(Base):
  __tablename__ = 'client'
  __table_args__ = {'autoload': True}
  user = relationship('User')

  @classmethod
  @cache.memoize(timeout=86400)
  def get_client(cls,client_id):
    return DBSession.query(cls).filter_by(client_id=client_id).first()

Then, in my view I have:

@oauth.clientgetter
def load_client(client_id):
  return Client.get_client(client_id)

But this gives the same result. I am using redis as my cache backend and the keys/values I have are:

1) "flask_cache_Pwd2uVDVikMYMDNB+gVWlW"
2) "flask_cache_api.models.Client.get_client_memver"
3) "flask_cache_http://lvho.st:5000/me"

GET flask_cache_Pwd2uVDVikMYMDNB+gVWlW:

"!ccopy_reg\n_reconstructor\np1\n(capi.models\nClient\np2\nc__builtin__\nobject\np3\nNtRp4\n(dp5\nS'_sa_instance_state'\np6\ng1\n(csqlalchemy.orm.state\nInstanceState\np7\ng3\nNtRp8\n(dp9\nS'manager'\np10\ng1\n(csqlalchemy.orm.instrumentation\n_SerializeManager\np11\ng3\nNtRp12\n(dp13\nS'class_'\np14\ng2\nsbsS'class_'\np15\ng2\nsS'modified'\np16\nI00\nsS'committed_state'\np17\n(dp18\nsS'instance'\np19\ng4\nsS'callables'\np20\n(dp21\nsS'key'\np22\n(g2\n(S'Iu6copdawXIQIskY5kwPgxFgU7JoE9lTSqmlqw29'\np23\nttp24\nsS'expired'\np25\nI00\nsbsVuser_id\np26\nL4L\nsVname\np27\nS'Default'\np28\nsV_default_scopes\np29\nS'email'\np30\nsVclient_id\np31\ng23\nsV_redirect_uris\np32\nS'http://localhost:8000/authorized/'\np33\nsVactive\np34\nI1\nsVclient_secret\np35\nS'Vnw0YJjgNzR06KiwXWmYz7aSPu1ht7JnY1eRil4s5vXLM9N2ph'\np36\nsVdescription\np37\nNsb."

GET flask_cache_api.models.Client.get_client_memver:

"!S'+gVWlW'\np1\n."

Can you please explain why you'd like to memoize your clientgetter? It doesn't seem like legit approach — shx2, Apr 08 '14 at 19:57

shx2 · Accepted Answer · 2014-04-08T20:01:58.987

Try reversing the order of your decorators:

@cache.memoize(timeout=86400)
@oauth.clientgetter
def load_client(client_id):
  return DBSession.query(Client).filter_by(client_id=client_id).first()

EDIT

The problem seem to be that a Client object is not pickle-able, while cache.memoize relies on objects' pickle-ability. Therefor, in one case, you end up with an invalid-client error (the client object did not "survive" the picke-dump-then-pickle-load process), and in another case, with some kind of caching error which (silently) prevents the object from being cached (I'm not sure what mechanism causes this silent-handling).

In any case, it seems to me you shouldn't attempt to memoize your client object in the first place.

Thanks. It doesn't throw an error but now it isn't caching. No key/value is created for it in redis either. — user3287829, Mar 09 '14 at 01:35

How can I cache/memoize my SQLAlchemy functions?

1 Answers1