Is there a way to make NuGet Package Source settings per solution?

TLDR: Yes

NuGet uses a hierarchical application of package sources starting with the NuGet.config at the level of your Windows User Profile and then applying more and more granular configuration starting at the root of the file path that contains your solution, finally ending with the directory containing your solution file.

So here's what I've managed to figure out - courtesy of a helpful Twitterer pointing me to this document:

https://docs.nuget.org/consume/nuget-config-file

When you edit the NuGet package sources in Visual Studio's Tools > NuGet Package Manager > Package Manager Settings: Package Sources option, it applies those changes by default to the NuGet.config file found in your %APPDATA%\NuGet directory. To override these settings on a per-solution (or per group of solutions) basis, you need to add a strategically placed NuGet.config file somewhere along the path of your solution or solutions.

All will become clear if you read the NuGet document, the solution I provide below will quickly allow you to specify a configuration for a single Visual Studio solution:

1. Navigate to %APPDATA%\NuGet and grab a copy of NuGet.config
2. Dump a copy in the root of your solution - i.e. where Application.sln lives.
3. Override the defaults applied to your user profile by editing the copy to contain only the NuGet package sources that are relevant for this solution - for instance, the private NuGet source that contains proprietary packages for this solution, but shouldn't be applied to other projects - for example:

<?xml version="1.0" encoding="utf-8"?>
<configuration>

  <packageRestore>
    <add key="enabled" value="True" />
    <add key="automatic" value="True" />
  </packageRestore>

  <activePackageSource>
    <add key="All" value="(Aggregate source)" />
  </activePackageSource>

  <packageSources>

    <!-- Ditch all the Global NuGet package sources we only want a 
         single private NuGet repo for this project -->
    <clear />

    <!-- Add the private NuGet package source for this solution -->
    <add key="My Private NuGet Server" value="http://myprivatenuget.com:8080/nuget" />

  </packageSources>

  <disabledPackageSources>

    <!-- Add any package sources to ignore here using the same keys as 
         defined in the packageSources list above-->

    <!--<add key="nuget.org" value="true" />-->

    <add key="Microsoft and .NET" value="true" />

  </disabledPackageSources>

</configuration>

If you want a configuration to apply to multiple solutions, ensure your solution folders are all contained within a common directory and put the NuGet.config for the package sources relevant for those solutions in that common directory, ensuring that any solution folders for projects that aren't to use these package sources are not contained in this common folder.

I want to add to the excellent answer provided by BenAlabaster. I had somewhat the opposite problem:

The company globally configured their custom private nuget feed for use on all solutions by default, and I wanted to make a "prototype" app using the public nuget feed.

With this (in the directory of that solution), the public nuget feed is available to my specific solution only, while keeping the company's feed the default for all other solutions:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <!-- Ditch all eventually upwards configured (private) feeds from an (enterprise) environment -->
    <clear />
    <!-- Make sure we use the public nuget -->
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
  </packageSources>
  <packageRestore>
    <add key="enabled" value="True" />
    <add key="automatic" value="True" />
  </packageRestore>
  <bindingRedirects>
    <add key="skip" value="False" />
  </bindingRedirects>
  <disabledPackageSources>
    <!-- Ditch all eventually upwards configured (private) feeds from an (enterprise) environment -->
    <clear />
  </disabledPackageSources>
</configuration>

The key was to clear all upwards disabled feeds, since they deliberately disabled the public feed in their NuGet.config in %APPDATA%\NuGet.