1

I found a weird behavior with cpython 2.5, 2.7, 3.2 and pypy with metaclass that override __new__ when using the python 2 / python 3 compatible way of using metaclass :

Given a module m1:

class C1Meta(type):
    def __new__(cls, name, bases, dct):
        return type.__new__(cls, name, bases, dct)

C1 = C1Meta('C1', (object,), {})


class C2Meta(type):
    pass

C2 = C2Meta('C2', (object,), {})

And the following main program:

import m1

C10 = m1.C1Meta('C10', (m1.C1,), {})


class C11Meta(m1.C1Meta):
    pass

C11 = C11Meta('C11', (m1.C1,), {})


class C12Meta(m1.C1Meta):
    def __new__(cls, name, bases, dct):
        return m1.C1Meta.__new__(cls, name, bases, dct)

C12 = C12Meta('C12', (m1.C1,), {})


class C13Meta(m1.C1Meta):
    def __new__(cls, name, bases, dct):
        return type.__new__(cls, name, bases, dct)

C13 = C13Meta('C13', (m1.C1,), {})


C20 = m1.C2Meta('C20', (m1.C2,), {})


class C21Meta(m1.C2Meta):
    pass

C21 = C21Meta('C21', (m1.C2,), {})


class C22Meta(m1.C2Meta):
    def __new__(cls, name, bases, dct):
        return m1.C2Meta.__new__(cls, name, bases, dct)

C22 = C22Meta('C22', (m1.C2,), {})


class C23Meta(m1.C2Meta):
    def __new__(cls, name, bases, dct):
        return type.__new__(cls, name, bases, dct)

C23 = C23Meta('C23', (m1.C2,), {})


print(C10)
print(C11)
print(C12)
print(C13)

print(C20)
print(C21)
print(C22)
print(C23)

Running the script will produce the following output (with all the mentioned python versions) :

<class 'm1.C10'>
<class 'm1.C11'>
<class 'm1.C12'>
<class '__main__.C13'>
<class '__main__.C20'>
<class '__main__.C21'>
<class '__main__.C22'>
<class '__main__.C23'>

-> the C10, C11 and C12 classes module is wrong !

Is it an expected behavior ?

Is there a way to override new that will not raise the issue ?

Thanks,

Christophe

Christophe de Vienne
  • 131
  • 6

1 Answers1

1

Apparently the __module__ attribute is set when the metaclass executes. In your case, the metaclass executes inside m1. With a normally defined class, the __module__ attribute is automatically generated and passed to the metaclass. However, you are creating your classes with a manual call to the metaclass, and passing in an empty attribute dictionary. Thus your classes do not provide a __module__ attribute. The __module__ attribute is created automatically, but not until type.__new__ is called. Since this happens inside module m1, the __module__ created at that time refers to m1.

I'm not exactly sure why you're calling the metaclass explicitly. The whole point of metaclasses is to allow you to customize what happens when you use a class statement to define a class. If you don't want to use a class statement, you can just use regular functions to create your classes, and don't need to bother with a metaclass at all.

That said, I believe you can get your desired behavior, by doing, e.g., 

C11 = C11Meta('C11', (m1.C1,), {'__module__': __name__})
BrenBarn
  • 242,874
  • 37
  • 412
  • 384
  • The thing is that if I do not override `__new__`, `type.__new__` is called and in this case the automatic `__module__` is right. It is weird. – Christophe de Vienne Oct 06 '12 at 18:50
  • As for why I am using the metaclass this way, it is because it is the only way that is python 2 and 3 compatible. – Christophe de Vienne Oct 06 '12 at 18:52
  • @ChristophedeVienne: What you say about `type.__new__` is true, probably because `type.__new__` is builtin so the usual notion of "what module it is in" doesn't apply. Unfortunately in Python you can't always rely on the base builtin behavior being identical to what you would get with an apparently no-op override of that behavior. – BrenBarn Oct 06 '12 at 18:53
  • @ChristophedeVienne: I don't know what you mean about Python 2 and 3 compatibility. What are you trying to do that doesn't work across versions? You're better off using the 2to3 converter than trying to cook up your own. – BrenBarn Oct 06 '12 at 18:55
  • I use metaclasses in a lib that works on both python 2 and 3 without using py2to3. The way I use metaclasses was found in http://python3porting.com/ (I am not cooking that kind of things as I want my code to be maintainable), but since then it has changed to advice the use of `six` to do that. I will go this way I think (need to try first though), although the workaround you proposed works (I have tested it). – Christophe de Vienne Oct 06 '12 at 19:02
  • Just tried six.with_metaclass (http://packages.python.org/six/#six.with_metaclass) that solve both the issue and the pain of defining my classes the way I did before. – Christophe de Vienne Oct 06 '12 at 19:07