pybind11 "Python is 64-bit, chosen compiler is 32-bit"

You should specify the 64-bit VS compiler like so:

cmake "/path/to/src/" -G"Visual Studio 14 2015 Win64"

Otherwise it selects the 32-bit by default.

If you are using the Ninja generator and you have this error make sure you run the VS Dev command prompt in 64-bit mode:

VsDevCmd.bat arch=amd64 && cmake <options> ...