Throwing immediately from async method

If you really want to do this, you can use the same approach Jon Skeet used in his reimplementation of LINQ: create a synchronous method that can throw or call the real asynchronous method:

public static Task TestExAsync(string filename)
{
    if (!System.IO.File.Exists(filename))
        throw new System.IO.FileNotFoundException(filename);

    return TestExAsyncImpl(filename);
}

private static async Task TestExAsyncImpl(string filename)
{
    await Task.Delay(1000);
}

Keep in mind that I think that it's normal to assume that a Task returning method doesn't throw directly. For example, you can use Task.WhenAll() to get all exceptions from several operations under normal circumstances, but this approach won't work when the exception is thrown immediately.

I think the normal behavior is appropriate. Your thread depends on the result of the async function to do its next processing, so the exception should be thrown on your thread. Your thread code could then take appropriate measure to recover from the exception. Because you could pass your Tasks around and initiate many Tasks, your recovery code could be in somewhere else where you need to get the Task result than your original called code. If the exception is thrown immediately, it could be thrown outside your recovery code.

The asynch void function throws immediately which makes sense because nothing depends on its result and there is no Task to pass around.

BTW, the point of exception handling is to recover the application state from exceptions, should not catch any exceptions that you cannot recover. When an exception is thrown, your application state may be corrupted, trying to continue working with corrupted applications cause more problems and security vulnerabilities.