Jupyter "500: Internal Server Error"; "ImportError: cannot import name ConverterMapping"

(answering my own question)

In my experience, these problems are usually caused by a compatibility issues between python packages installed via apt, and those installed via pip.

Unfortunately, I could not reconstruct the lengthy series of steps taken to install the jupyter/scipy/etc environment. Some packages must have been pulled in via apt and others via pip, leading to version conflicts.

This approach fixed the problem initially:

• Following this answer, I ran python -c "import site; print(site.USER_BASE)" to find where pip installs --user packages
• For me, this was ~/.local
• I moved ~/.local to ~/.local_disabled to eliminate all locally installed packages
• At this point, Jupyter worked again, (it must have been installed with the system python via apt).
• I've added only a minimal subset of user packages, and will keep an eye out for incompatibilities.
• I imagine avoiding pulling in Jupyter/ipython via apt might be a better way to avoid this problem in the future.

After subsequent system updates, the issue returned

This seems to confirm that the problem arises due to a mis-match between python packages installed via apt and those installed via pip. This time, to fix the issue, I:

• Uninstalled python packages that were installed via apt (but not part of the core system installed with Ubuntu). This mainly included jupyter, and many of its dependencies.
• Reinstalled Jupyter using pip install jupyter --user.
• This is working (for now).

( Please feel free to edit/comment/extend this answer. )

I had a similar experience and found Mrule's answer. The deleting/renaming .local worked for me too but his long term solution didn't sadly. So I dug a little deep to find why the first solution worked.

Turns out which jupyter returned $HOME/.local/bin/jupyter when .local was not deleted and /usr/bin/jupyter when I did. So the problem was in the pip packages (the ones installed in .local by the pip install --user jupyter option).

I had previously did sudo pip uninstall jupyter several times before, but that doesn't remove the packages in .local (see here). You have to do pip uninstall jupyter without sudo to uninstall those in the home directory (or manually delete them). I uninstalled any other jupyter related packages in my system by finding them using pip list | grep jupyter and apt list | grep jupyter.

And then finally did a fresh install of jupyter via sudo apt install jupyter. And now it seems to be working.

Bottom line is that the packages installed via pip system wide and in the home directory (.local) and that installed via apt were conflicting somehow (I couldn't find exactly why).

The safest solution to this is to create a virtual environment and run your jupyter from it. This clearly separates all the operations of the pip and apt. It worked for me. I use Ubuntu

To create a virtual environment, run :

1)sudo pip install --upgrade virtualenv (install virtualenv)

2)virtualenv  xyz --python=python3.7  (xyz is the name of the new virtual environment)

3) cd into the directory of xyz

4) source ./bin/activate 

5)pip install jupyter