How to instantiate or mock a Window programmatically?

Short answer is that you shouldn't. Unit Testing is not done on UI.

Unit Tests should be run on your logical code, and that's where patterns like MVC and MVVM come to help. The UI should be tested using tools like CodedUI.

if your logic has a dependency on UI, then you are doing it wrong.

Try the following:

if(Application.ResourceAssembly == null)
    Application.ResourceAssembly = typeof(MainWindow).Assembly;
var window = new MainWindow();