Does Python have a package/module management system?

As a Ruby and Perl developer and learning-Python guy, I haven't found easy_install or pip to be the equivalent to RubyGems or CPAN.

I tend to keep my development systems running the latest versions of modules as the developers update them, and freeze my production systems at set versions. Both RubyGems and CPAN make it easy to find modules by listing what's available, then install and later update them individually or in bulk if desired.

easy_install and pip make it easy to install a module ONCE I located it via a browser search or learned about it by some other means, but they won't tell me what is available. I can explicitly name the module to be updated, but the apps won't tell me what has been updated nor will they update everything in bulk if I want.

So, the basic functionality is there in pip and easy_install but there are features missing that I'd like to see that would make them friendlier and easier to use and on par with CPAN and RubyGems.


The Python Package Index (PyPI) seems to be standard:

  • To install a package: pip install MyProject
  • To update a package pip install --upgrade MyProject
  • To fix a version of a package pip install MyProject==1.0

You can install the package manager as follows:

curl -O http://python-distribute.org/distribute_setup.py
python distribute_setup.py
easy_install pip

References:

  • http://guide.python-distribute.org/
  • http://pypi.python.org/pypi/distribute

And just to provide a contrast, there's also pip.


Recent progress

March 2014: Good news! Python 3.4 ships with Pip. Pip has long been Python's de-facto standard package manager. You can install a package like this:

pip install httpie

Wahey! This is the best feature of any Python release. It makes the community's wealth of libraries accessible to everyone. Newbies are no longer excluded from using community libraries by the prohibitive difficulty of setup.

However, there remains a number of outstanding frustrations with the Python packaging experience. Cumulatively, they make Python very unwelcoming for newbies. Also, the long history of neglect (ie. not shipping with a package manager for 14 years from Python 2.0 to Python 3.3) did damage to the community. I describe both below.

Outstanding frustrations

It's important to understand that while experienced users are able to work around these frustrations, they are significant barriers to people new to Python. In fact, the difficulty and general user-unfriendliness is likely to deter many of them.

PyPI website is counter-helpful

Every language with a package manager has an official (or quasi-official) repository for the community to download and publish packages. Python has the Python Package Index, PyPI. https://pypi.python.org/pypi

Let's compare its pages with those of RubyGems and Npm (the Node package manager).

  1. https://rubygems.org/gems/rails RubyGems page for the package rails
  2. https://www.npmjs.org/package/express Npm page for the package express
  3. https://pypi.python.org/pypi/simplejson/ PyPI page for the package simplejson

You'll see the RubyGems and Npm pages both begin with a one-line description of the package, then large friendly instructions how to install it.

Meanwhile, woe to any hapless Python user who naively browses to PyPI. On https://pypi.python.org/pypi/simplejson/ , they'll find no such helpful instructions. There is however, a large green 'Download' link. It's not unreasonable to follow it. Aha, they click! Their browser downloads a .tar.gz file. Many Windows users can't even open it, but if they persevere they may eventually extract it, then run setup.py and eventually with the help of Google setup.py install. Some will give up and reinvent the wheel..

Of course, all of this is wrong. The easiest way to install a package is with a Pip command. But PyPI didn't even mention Pip. Instead, it led them down an archaic and tedious path.

Error: Unable to find vcvarsall.bat

Numpy is one of Python's most popular libraries. Try to install it with Pip, you get this cryptic error message:

Error: Unable to find vcvarsall.bat

Trying to fix that is one of the most popular questions on Stack Overflow: "error: Unable to find vcvarsall.bat"

Few people succeed.

For comparison, in the same situation, Ruby prints this message, which explains what's going on and how to fix it:

Please update your PATH to include build tools or download the DevKit from http://rubyinstaller.org/downloads and follow the instructions at http://github.com/oneclick/rubyinstaller/wiki/Development-Kit

Publishing packages is hard

Ruby and Nodejs ship with full-featured package managers, Gem (since 2007) and Npm (since 2011), and have nurtured sharing communities centred around GitHub. Npm makes publishing packages as easy as installing them, it already has 64k packages. RubyGems lists 72k packages. The venerable Python package index lists only 41k.

History

Flying in the face of its "batteries included" motto, Python shipped without a package manager until 2014.

Until Pip, the de facto standard was a command easy_install. It was woefully inadequate. The was no command to uninstall packages.

Pip was a massive improvement. It had most the features of Ruby's Gem. Unfortunately, Pip was--until recently--ironically difficult to install. In fact, the problem remains a top Python question on Stack Overflow: "How do I install pip on Windows?"

Tags:

Python

Module

Recent Posts