Current state of python namespace packages

The Python documentation has a good description of the three ways of creating namespace packages in Python, including guidelines for when to use each of the three methods. Furthermore, this topic is discussed in great depth in a different StackOverflow thread which has a good accepted answer. Finally, if you are someone who would rather read code than documentation, the sample-namespace-packages repo contains examples of namespace packages created using each of the three available methods.

In brief, if you intend your packages to work with Python versions 3.3 and above, you should use the native namespace packages method. If you intend your packages to work with older versions of Python, you should use the pkgutil method. If you intend to add a namespace package to a namespace that is already using the pkg_resources method, you should continue to use method.

---

With native namespace packages, we can remove __init__.py from both packages and modify our setup.py files to look as follows:

# setup.py file for my_data.source_a
from setuptools import setup, find_namespace_packages

setup(
    name="my_data.source_a",
    version="0.1",
    packages=find_namespace_packages(include=['my_data.*'])
)

# setup.py file for my_data.source_b
from setuptools import setup, find_namespace_packages

setup(
    name="my_data.source_b",
    version="0.1",
    packages=find_namespace_packages(include=['my_data.*'])
)

We need to add the include=['my_data.*'] argument because, by default find_namespace_packages() is rather lenient in the folders that it includes as namespace packages, as described here.

This is the recommended approach for packages supporting Python 3.3 and above.

---

With pkgutil-style namespace packages, we need to add the following line to the my_data.__init__.py files in each of our packages:

__path__ = __import__('pkgutil').extend_path(__path__, __name__)

This is the approach used by the backports namespace, and by different packages in the google-cloud-python repo, and it is the recommended approach for supporting older versions of Python.

The latest version of Python which is Python 3.7 uses the native namespace packages approach to create namespace packages which are defined in PEP 420.

There are currently three different approaches to creating namespace packages:

1. Use native namespace packages. This type of namespace package is defined in PEP 420 and is available in Python 3.3 and later. This is recommended if packages in your namespace only ever need to support Python 3 and installation via pip.
2. Use pkgutil-style namespace packages. This is recommended for new packages that need to support Python 2 and 3 and installation via both pip and python setup.py install.
3. Use pkg_resources-style namespace packages. This method is recommended if you need compatibility with packages already using this method or if your package needs to be zip-safe.

Reference: Packaging namespace packages