What's the point of passing ExceptionDispatchInfo around instead of just the Exception?

You're assuming that exceptions are immutable. This is not the case - an exception's StackTrace changes when it's re-thrown.

The purpose of ExceptionDispatchInfo.Capture is to capture a potentially mutating exception's StackTrace at a point in time:

void Foo() => throw new InvalidOperationException ("foo");

Exception original = null;
ExceptionDispatchInfo dispatchInfo = null;
try
{
    try
    {
        Foo();
    }
    catch (Exception ex)
    {
        original = ex;
        dispatchInfo = ExceptionDispatchInfo.Capture (ex);
        throw ex;
    }
}
catch (Exception ex2)
{
    // ex2 is the same object as ex. But with a mutated StackTrace.
    Console.WriteLine (ex2 == original);  // True
}

// So now "original" has lost the StackTrace containing "Foo":
Console.WriteLine (original.StackTrace.Contains ("Foo"));  // False

// But dispatchInfo still has it:
try
{
    dispatchInfo.Throw ();
}
catch (Exception ex)
{
    Console.WriteLine (ex.StackTrace.Contains ("Foo"));   // True
}

ExceptionDispatchInfo is used to preserve the stack trace after an Exception is thrown, allowing you to catch that exception, not throwing it immediately (as part of a catch), and to raise such exception on a later point in the future.

I found a good example of this on https://thorarin.net/blog/post/2013/02/21/Preserving-Stack-Trace.