Debug dynamically loaded assembly in Visual Studio .NET
Build the source locally of dynamically loaded assembly and make sure you have the PDB files where the referencing app is running.
This is a common requirement for people wishing to debug the ASP.NET MVC source code. There are some caveats regarding the GAC though.
Another quick (but dirty) way to do is to temporarily add the project for the dynamically loaded assembly to your solution. This is what Steve Sanderson recommends for debugging the ASP.NET MVC framework. Personally I prefer the PDB root.
Do you have the PDB files colocated with the DLLs, having been built from the sources in their current locations?