fatal error: mpi.h: No such file or directory #include <mpi.h>

On my system, I was just missing the Linux package.

sudo apt install libopenmpi-dev
pip install mpi4py

(example of something that uses it that is a good instant test to see if it succeeded)

Succeded.

The problem is almost certainly that you're not using the MPI compiler wrappers. Whenever you're compiling an MPI program, you should use the MPI wrappers:

• C - mpicc
• C++ - mpiCC, mpicxx, mpic++
• FORTRAN - mpifort, mpif77, mpif90

These wrappers do all of the dirty work for you of making sure that all of the appropriate compiler flags, libraries, include directories, library directories, etc. are included when you compile your program.