Dict-like object with no getitem, setitem

Question

If you do

import gdbm
db = gdbm.open('foo', 'cs')

You get an object which is:

<gdbm.gdbm at 0x7f0982b9aef0>

You can now set keys and values in the database via:

db['foo'] = 'bar'
print db['foo']

I wanted to use these from Twisted and make a wrapper for __getitem__ and __setitem__ which returns deferreds. However, I noticed something peculiar:

In [42]: dir(db)
Out[42]: ['close', 'firstkey', 'has_key', 'keys', 'nextkey', 'reorganize', 'sync']

This object hasn't got __getitem__ and __setitem__. Trying either of these gives an attribute access error. Yet, it behaves like a dictionary. What sort of object is this?

(I suspect this is a C extension object, but I find it odd that it has dict-like access methods, but no __getitem__ and __setitem__ methods. A pointer to Python doc describing this behavior would be helpful.)

Further: how would you get references to db.__getitem__ and db.__setitem__ if you wanted to wrap them in a deferred? The only solution I see is to wrap db in a utility class:

class Db:
    def __init__(self, db):
         self.db = db

    def __getitem__(self, x):
         return self.db[x]

    ...

But perhaps I am missing something obvious?

@Jean-PaulCalderone: I meant writing something of the spirit `class Db: def get(self, key): return deferToThread(db.__getitem__, key)`. Performance isn't an issue here, but I would be very interested to know what you think of accessing gdbm from Twisted in that manner. — Max, Oct 07 '13 at 00:19
In my experience, turning syntactic constructs like `db[key]` into implicitly threaded (or otherwise complicated, long-running) operations leads to more confusion than it's ultimately worth. Also, keep in mind that `__setitem__` isn't even allowed to return a value - so the `Deferred` representing the completion of that operation in a thread is lost (so, for example, you have no way to do error handling). — Jean-Paul Calderone, Oct 07 '13 at 10:11
@Jean-PaulCalderone: thanks. Yes, explicit is better, I have it written now as `d = db.set(key, value)` and `d = db.get(key)`. It works very nicely and looks decent in `inlineCallback`-ed functions. — Max, Oct 07 '13 at 14:42

Martijn Pieters · Accepted Answer · 2013-10-07T08:03:04.900

gdbm is indeed a C module; the object implements the C-API mapping protocol instead.

See the source code for the PyMappingMethods structure with function pointers.

You could use operator.getitem() to access the keys:

>>> from operator import getitem
>>> db['foo'] = 'bar'
>>> getitem(db, 'foo')
'bar'

You could wrap getitem() in functools.partial() to make it an efficient callable with one argument:

>>> from functools import partial
>>> gi = partial(getitem, db)
>>> gi('foo')
'bar'

As getitem and partial are implemented in C, this combo will always be faster than custom objects, functions or lambdas to wrap the access.

Dict-like object with no __getitem__, __setitem__

1 Answers1

Dict-like object with no getitem, setitem