Python Metaclass : Understanding the 'with_metaclass()'

with_metaclass() is a utility class factory function provided by the six library to make it easier to develop code for both Python 2 and 3.

It uses a little sleight of hand (see below) with a temporary metaclass, to attach a metaclass to a regular class in a way that's cross-compatible with both Python 2 and Python 3.

Quoting from the documentation:

Create a new class with base class base and metaclass metaclass. This is designed to be used in class declarations like this:

from six import with_metaclass
   
class Meta(type):
    pass

class Base(object):
    pass

class MyClass(with_metaclass(Meta, Base)):
    pass

This is needed because the syntax to attach a metaclass changed between Python 2 and 3:

Python 2:

class MyClass(object):
    __metaclass__ = Meta

Python 3:

class MyClass(metaclass=Meta):
    pass

The with_metaclass() function makes use of the fact that metaclasses are a) inherited by subclasses, and b) a metaclass can be used to generate new classes and c) when you subclass from a base class with a metaclass, creating the actual subclass object is delegated to the metaclass. It effectively creates a new, temporary base class with a temporary metaclass metaclass that, when used to create the subclass swaps out the temporary base class and metaclass combo with the metaclass of your choice:

def with_metaclass(meta, *bases):
    """Create a base class with a metaclass."""
    # This requires a bit of explanation: the basic idea is to make a dummy
    # metaclass for one level of class instantiation that replaces itself with
    # the actual metaclass.
    class metaclass(type):

        def __new__(cls, name, this_bases, d):
            return meta(name, bases, d)

        @classmethod
        def __prepare__(cls, name, this_bases):
            return meta.__prepare__(name, bases)
    return type.__new__(metaclass, 'temporary_class', (), {})

Breaking the above down:

• type.__new__(metaclass, 'temporary_class', (), {}) uses the metaclass metaclass to create a new class object named temporary_class that is entirely empty otherwise. type.__new__(metaclass, ...) is used instead of metaclass(...) to avoid using the special metaclass.__new__() implementation that is needed for the slight of hand in a next step to work.
• In Python 3 only, when temporary_class is used as a base class, Python first calls metaclass.__prepare__() (passing in the derived class name, (temporary_class,) as the this_bases argument. The intended metaclass meta is then used to call meta.__prepare__(), ignoring this_bases and passing in the bases argument.
• next, after using the return value of metaclass.__prepare__() as the base namespace for the class attributes (or just using a plain dictionary when on Python 2), Python calls metaclass.__new__() to create the actual class. This is again passed (temporary_class,) as the this_bases tuple, but the code above ignores this and uses bases instead, calling on meta(name, bases, d) to create the new derived class.

As a result, using with_metaclass() gives you a new class object with no additional base classes:

>>> class FooMeta(type): pass
...
>>> with_metaclass(FooMeta)  # returns a temporary_class object
<class '__main__.temporary_class'>
>>> type(with_metaclass(FooMeta))  # which has a custom metaclass
<class '__main__.metaclass'>
>>> class Foo(with_metaclass(FooMeta)): pass
...
>>> Foo.__mro__  # no extra base classes
(<class '__main__.Foo'>, <type 'object'>)
>>> type(Foo) # correct metaclass
<class '__main__.FooMeta'>

UPDATE: the six.with_metaclass() function has since been patched with a decorator variant, i.e. @six.add_metaclass(). This update fixes some mro issues related to the base objects. The new decorator would be applied as follows:

import six

@six.add_metaclass(Meta)
class MyClass(Base):
    pass

Here are the patch notes and here is a similar, detailed example and explanation for using a decorator alternative.