Weird inheritance with metaclasses

Question

I'm experiencing some really weird problems in Python when trying to inherit from a class with a metaclass. I have this:

class NotifierMetaclass(type):

    def __new__(cls, name, bases, dct):

        attrs = ((name, value) for name, value in dct.items()
                    if not name.startswith('__'))

        def wrap_method(meth):
            return instance_wrapper()(meth) # instance_wrapper is a decorator of my own

        def is_callable(value):
            return hasattr(value, '__call__')

        decorated_meth = dict(
            (name, value) if not is_callable(value)
            else (name, wrap_method(value))
            for name, value in attrs
        )

        return super(NotifierMetaclass, cls).__new__(
            cls, name, bases, decorated_meth
        )


class Notifier(object):

    def __init__(self, instance):
        self._i = instance

    __metaclass__ = NotifierMetaclass

And then, in notifiers.py:

from helpers import Notifier

class CommentNotifier(Notifier):

    def __notification__(self, notification):
        return '%s has commented on your board' % self.sender

    def __notify__(self):
        receivers = self.retrieve_users()
        notif_type = self.__notificationtype__()
        for user in receivers:
            Notification.objects.create(
                object_id=self.id,
                receiver=user,
                sender_id=self.sender_id,
                type=notif_type
            )

However, when I try to import CommentNotifier it returns Notifier. In the shell:

$ python
Python 2.7.3 (default, Apr 20 2012, 22:44:07) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from logic.notifiers import CommentNotifier
>>> CommentNotifier
<class 'helpers.CommentNotifier'>

In fact, this is (at least that's what I think) actually the same problem I got a week ago with some Django models. At first I thought it was related to the way Django works, but now I suspect it's more like a Python "problem" with metaclasses and inheritance.
Is this a known issue or am I simply doing things wrong? Hope you can help me.
EDIT: I forgot to mention that I attribute this "error" to metaclasses because if I don't give a metaclass to Notifier it works as expected.

Answer 1

Okay, I think I figured it out. The correct class is being imported. It just has the wrong name. You should be able to see this if you set attributes on the classes. If you put someJunk = "Notifier" in the Notifier definition and someJunk = "CommentNotifier" in the CommentNotifier definition, then when you import CommentNotifier it will have the right value.

The problem is that in creating your attrs, you exclude all double-underscore attributes, including __module__. When you call the superclass __new__, you pass in your attrs, which does not have a __module__ entry, so Python creates one for you. But since this code is executing inside the file containing the metaclass, the module is incorrectly set to the metaclass's file and not the actual class's file.

I'm not seeing the behavior you observe for the actual name of the class, only for the module. That is, for me the imported class is named metafile.CommentNotifier, where metafile is the file containing the metaclass. It should be named submeta.CommentNotifier, where submeta is the file containing the CommentNotifierClass. I'm not sure why you're seeing it for __name__ as well, but it wouldn't surprise me if some subtle handling of module/name assignment varies across different Python versions.

__notify__ and __notification__ are not Python magic methods. It appears that you're excluding double-underscore methods because you are using double underscores to indicate something for your own purposes. You shouldn't do this. Use some other prefix for your own methods (like _Notifier or something) if you must, then exclude those methods and leave the double-underscore ones alone. Excluding double-underscore methods could cause other problems. In particular, it will cause failure if you ever decide to define a real magic method (e.g., __str__) on a class that uses this metaclass.

(To clarify: you can use methods that begin with double underscores if you want, as private attributes, although this still is probably not a good idea. If you do this, though, you need to make sure you only do your special processing on those attributes, and not ones that begin and end with double underscores, which are Python-internal magic methods. What you shouldn't do is create your own names that begin and end with double underscores, like __notify__.)