Is there a Visual Studio Build Profiler?

You can use MSBuild Profiler to get a breakdown of how long each project and it's tasks are taking. It's a little old and doesn't seem to be supported but I've still found it very useful. Download and install from here then run as follows from the VS command line:

MSBuild.exe MySolution.sln /l:MSBuildProfileLogger,MSBuildProfiler,Version=1.0.1.0,Culture=neutral,PublicKeyToken=09544254e89d148c

When it finishes it will pop-up a GUI with a tree structure breakdown of your solution, containing the time it took to process each project and task.

@PostMan's answer is what you need (hence my +1 on that)

The MSBuild (4.0) debugger may also prove useful

Tools -> Options... -> Projects and Solutions -> Build And Run

Then change MSBuild project build output verbosity

When on normal (defaul is minimal) it gives timings for each project, so this could be used to pin point what is causing the slow down. (Assuming you use the Output window)

Or adjust to a higher level to see more detail of what is going on during a compile.