How do I get all instances of all loaded types that implement a given interface?

I don't believe there is a way... You would have to either be able to walk the Heap, and examine every object there, or walk the stack of every active thread in the application process space, examining every stack reference variable on every thread...

The other way, (I am guessing you can't do) is intercept all Object creation activities (using a container approach) and keep a list of all objects in your application...

If you need instances (samples) of all types implementing particular interface you can go through all types, check for interface and create instance if match found.

Here's some pseudocode that looks remarkably like C# and may even compile and return what you need. If nothing else, it will point you in the correct direction:

public static IEnumerable<T> GetInstancesOfImplementingTypes<T>()
{
    AppDomain app = AppDomain.CurrentDomain;
    Assembly[] ass = app.GetAssemblies();
    Type[] types;
    Type targetType = typeof(T);

    foreach (Assembly a in ass)
    {
        types = a.GetTypes();
        foreach (Type t in types)
        {
            if (t.IsInterface) continue;
            if (t.IsAbstract) continue;
            foreach (Type iface in t.GetInterfaces())
            {
                if (!iface.Equals(targetType)) continue;
                yield return (T) Activator.CreateInstance(t);
                break;
            }
        }
    }
}

Now, if you're talking about walking the heap and returning previously instantiated instances of all objects that implement a particular type, good luck on that as this information is not stored by .Net runtime (can be computed by debugers/profilers by examining heap/stack so).

Depending on the reason why you think you need to do that there are probably better ways of going about it.