Namespace vs regular package

Reading link from Aaron, and PEP420, it appears that the fundamental difference between a namespace package and a regular package, beside the obvious difference that a regular package may contain various initialization code in __init__.py, is that a namespace package is a virtual package whose contents can be distributed in various places along Python's lookup path.

For example, given

a/foo/bar.py
b/foo/baz.py

If both b and a are in Python's path, you can import foo.bar and foo.baz freely.

Of course, this begs the question that, if __init__.py is not needed, then all other things being equal, is it better to make a regular package or a namespace package, but is a little off-topic.

Namespace packages

As of Python 3.3, we get namespace packages. These are a special kind of package that allows you to unify two packages with the same name at different points on your Python-path. For example, consider path1 and path2 as separate entries on your Python-path:

path1
+--namespace
   +--module1.py
   +--module2.py
path2
+--namespace
   +--module3.py
   +--module4.py

with this arrangement you should be able to do the following:

from namespace import module1, module3

thus you get the unification of two packages with the same name in a single namespace.

If either one of them gain an __init__.py that becomes the package - and you no longer get the unification as the other directory is ignored.

If both of them have an __init__.py, the first one in the PYTHONPATH (sys.path) is the one used.

__init__.py used to be required to make directory a package

Namespace packages are packages without the __init__.py.

For an example of a simple package, if you have a directory:

root
+--package
   +--file1.py
   +--file2.py
   ...

While you could run these files independently in the package directory, e.g. with python2 file1.py, under Python 2 you wouldn't be able to import the files as modules in the root directory, e.g.

import package.file1

would fail, and in order for it to work, you at least need this:

package
  +--__init__.py
  +--file1.py
  +--file2.py
  ...

__init__.py initializes the package so you can have code in the __init__.py that is run when the module is first imported:

run_initial_import_setup()

provide an __all__ list of names to be imported,

__all__ = ['star_import', 'only', 'these', 'names']

if the package is imported with the following:

from module import *

or you can leave the __init__.py completely empty if you only want to be able to import the remaining .py files in the directory.

Namespaces with __init__.py using pkgutil:

You could originally use pkgutil, available since Python 2.3. to accomplish adding namespaces, by adding the following into each separate package's __init__.py:

from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)

Setuptools uses a similar method, again, all __init__.py files should contain the following (with no other code):

import pkg_resources
pkg_resources.declare_namespace(__name__)

Namespaces were more thoroughly addressed in PEP 420

See also more discussion on setuptools and Namespaces here:

http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages