Where are the settings for multiple startup projects on a solution stored?

I believe it is in the Visual Studio User Solution User Options file *.suo. This file is hidden by default. Most source control providers ignore this file by default because it stores location specific paths.

You can install the SwitchStartupProject extension. This will create a file called YourSolution.startup.suo containing the startup projects, which you can check into source control.

See also their wiki for configuration instructions and other info.

According @dan-gpd answer, please use the following updated extension: SwitchStartupProject for VS 2019