pip 10 and apt: how to avoid "Cannot uninstall X" errors for distutils packages

This is what worked for me--

pip install --ignore-installed <Your package name>

or

sudo pip install --ignore-installed <Your package name>

or (inside jupyter notebook)

import sys
!{sys.executable} -m pip install --ignore-installed <Your package name>

This is the solution I ended up going with, and our apps have been running in production without any issues for close to a month with this fix in place:

All I had to do was to add

--ignore-installed

to the pip install lines in my dockerfile that were raising errors. Using the same dockerfile example from my original question, the fixed dockerfile would look something like:

FROM ubuntu:14.04

RUN apt-get -y update && apt-get -y install \
    python-pip \
    python-numpy # ...and many other packages

RUN pip install -U pip

RUN pip install -r /tmp/requirements1.txt --ignore-installed # don't try to uninstall existing packages, e.g., numpy
RUN pip install -r /tmp/requirements2.txt
RUN pip install -r /tmp/requirements3.txt

The documentation I could find for --ignore-installed was unclear in my opinion (pip install --help simply says "Ignore the installed packages (reinstalling instead)."), and I asked about the potential dangers of this flag here, but have yet to get satisfying answer. However, if there are any negative side effects, our production environment has yet to see the effects of them, and I think the risk is low/none (at least that has been our experience). I was able to confirm that in our case, when this flag was used, the existing installation was not uninstalled, but that the newer installation was always used.

Update:

I wanted to highlight this answer by @ivan_pozdeev. He provides some information that this answer does not include, and he also outlines some potential side-effects of my solution.