MSBuild - How to build multiple files AND projects in parallel

The /MP switch for the compiler (cl.exe) and the /m option for msbuild.exe are indeed the way to go. Here are some relevant documentation extracts:

The /MP option causes the compiler to create one or more copies of itself, each in a separate process. Then these copies simultaneously compile the source files. Optional Argument: The maximum number of processes that the compiler can create. If you omit the processMax argument, the compiler retrieves the number of effective processors on your computer from the operating system, and creates a process for each processor. The /MP option is incompatible with some compiler options and language features. If you use an incompatible compiler option with the /MP option, the compiler issues warning D9030 and ignores the /MP option

Examples suppose you specify the following command line:

cl /MP7 a.cpp b.cpp c.cpp d.cpp e.cpp

In this case the compiler uses five processes because that is the lesser of five source files and a maximum of seven processes. Alternatively, suppose your computer has two effective processors and you specify the following command line:

cl /MP a.cpp b.cpp c.cpp

In this case the operating system reports two processors; therefore, the compiler uses two processes in its calculation. As a result, the compiler will execute the build with two processes because that is the lesser of two processes and three source files.

So as you noticed this was all about the compiler, and no word about msbuild yet, since the compiler is a standalone tool. In other words: suppose you open 2 command windows and in both of them invoke a command like cl /MP3 a.cpp b.cpp c.cpp at the same time, you'd have 6 compiler processes running at the same time. Which is the behaviour you are after and which is also what msbuild can do.

Output from invoking msbuild /?:

/maxcpucount[:n] Specifies the maximum number of concurrent processes to build with. If the switch is not used, the default value used is 1. If the switch is used without a value MSBuild will use up to the number of processors on the computer. (Short form: /m[:n])

Also more relevant information here, specifically the part where it says

BuildInParallel is an optional boolean parameter on a MSBuild task. When BuildInParallel is set to true (its default value), multiple worker processes are generated to build as many projects at the same time as possible. For this to work correctly, the /maxcpucount switch must be set to a value greater than 1, and the system must be at least dual-core or have two or more processors.

Now if you'd just call msbuild /m my.vcxproj on a typical project this won't do anything special because there is only one project to build. However if inside an msbuild file there is an call to the MsBuild task like

<MsBuild Projects="@(ListOfProjects)" BuildInParallel="True"/>

and the /m option is passed to msbuild then msbuild will spawn multiple other msbuild processes per the rules laid out above. And coincidently that is exactly what happens when building a solution with msbuild. Msbuild first converts the solution to an actual msbuild file which calls the MsBuild task with a list of projects in the solution. As such multiple projects can be built in parallel and for that multiple msbuild processes are created. In turn, if these are C++ projects with the /MP option set, each of those projects will create multiple compiler processes yielding a maximum of <max num cl processes> * <max num msbuild instances> parallel compilations. It depends on your build machine what is the optimum, if you want you can play with the options (to do this efficiently each of your projects would import a common property sheet in which you set the /MP option else you'd have to edit all of them seperately) and look at the elapsed time msbuild reports when a build is done. I always just leave the arguments for /m and /MP to the default because I don't want other developpers using my projects to have possible suboptimal configurations, and because it easily maxes out CPU.

You can achieve this on the command line by calling:

MSBuild /m:PARALLEL_PROJECT_COUNT /p:CL_MPCount=PARALLEL_FILE_COUNT

The number of resulting processes is going to be:

MSBuild = PARALLEL_PROJECT_COUNT Cl = PARALLEL_PROJECT_COUNT * PARALLEL_FILE_COUNT