I want await to throw AggregateException, not just the first Exception

Here is a shorter implementation of Stephen Cleary's WithAggregateException extension method:

public static async Task WithAggregateException(this Task source)
{
    try { await source.ConfigureAwait(false); }
    catch (OperationCanceledException) when (source.IsCanceled) { throw; }
    catch { source.Wait(); }
}

public static async Task<T> WithAggregateException<T>(this Task<T> source)
{
    try { return await source.ConfigureAwait(false); }
    catch (OperationCanceledException) when (source.IsCanceled) { throw; }
    catch { return source.Result; }
}

This approach is based on a suggestion by Stephen Toub in this API proposal in GitHub.

---

Update: I added a special handling of the cancellation case, to prevent the awkwardness of propagating an AggregateException that contains an OperationCanceledException. Now the OperationCanceledException is propagated directly, and the Task.IsCanceled status is preserved. Kudos to @noseratio for pointing out this flaw in the comments of this answer. Of course now this implementation is not much shorter than Stephen Cleary's approach!

I disagree with the implication in your question title that await's behavior is undesired. It makes sense in the vast majority of scenarios. In a WhenAll situation, how often do you really need to know all of the error details, as opposed to just one?

The main difficulty with AggregateException is the exception handling, i.e., you lose the ability to catch a particular type.

That said, you can get the behavior you want with an extension method:

public static async Task WithAggregateException(this Task source)
{
  try
  {
    await source.ConfigureAwait(false);
  }
  catch
  {
    // source.Exception may be null if the task was canceled.
    if (source.Exception == null)
      throw;

    // EDI preserves the original exception's stack trace, if any.
    ExceptionDispatchInfo.Capture(source.Exception).Throw();
  }
}

Exception Handling (Task Parallel Library)

I could say more but it would just be padding. Play with it, it does work as they say. You just have to be careful.

maybe you want this

God (Jon Skeet) explains await exception handling

(personally i shy away from await, but thats just my preference)

in response to comments (too long for a comment reply)

Then use threads as your starting point for an analogous argument as the best practises there will be the source of ones for here.

Exceptions happily get swallowed unless you implement code to pass them out (for instance the async pattern that the await is preumably wrapping ... you add them to an event args object when you raise an event). When you have a scenario where you fire up an arbitrary number of threads and execute on them you have no control over order or the point at which you terminate each thread. Moreover you would never use this pattern if an error on one was relevant to another. Therefor you are strongly implying that execution of the rest is completley independent - IE you are strongly implying that exceptions on these threads have already been handled as exceptions. If you want to do something beyond handling exceptions in these threads in the threads they occur in (which is bizzarre) you should add them to a locking collection that is passed in by reference - you are no longer considering exceptions as exceptions but as a piece of information - use a concurrent bag, wrap the exception in the info you need to identify the context it came from - which would of been passed into it.

Don't conflate your use cases.

I know I'm late but i found this neat little trick which does what you want. Since the full set of exceptions are available with on awaited Task, calling this Task's Wait or a .Result will throw an aggregate exception.

    static void Main(string[] args)
    {
        var task = Run();
        task.Wait();
    }
    public static async Task Run()
    {

        Task[] tasks = new[] { CreateTask("ex1"), CreateTask("ex2") };
        var compositeTask = Task.WhenAll(tasks);
        try
        {
            await compositeTask.ContinueWith((antecedant) => { }, TaskContinuationOptions.ExecuteSynchronously);
            compositeTask.Wait();
        }
        catch (AggregateException aex)
        {
            foreach (var ex in aex.InnerExceptions)
            {
                Console.WriteLine(ex.Message);
            }
        }
    }

    static Task CreateTask(string message)
    {
        return Task.Factory.StartNew(() => { throw new Exception(message); });
    }