Why can't I use the 'await' operator within the body of a lock statement?

I assume this is either difficult or impossible for the compiler team to implement for some reason.

No, it is not at all difficult or impossible to implement -- the fact that you implemented it yourself is a testament to that fact. Rather, it is an incredibly bad idea and so we don't allow it, so as to protect you from making this mistake.

call to Monitor.Exit within ExitDisposable.Dispose seems to block indefinitely (most of the time) causing deadlocks as other threads attempt to acquire the lock. I suspect the unreliability of my work around and the reason await statements are not allowed in lock statement are somehow related.

Correct, you have discovered why we made it illegal. Awaiting inside a lock is a recipe for producing deadlocks.

I'm sure you can see why: arbitrary code runs between the time the await returns control to the caller and the method resumes. That arbitrary code could be taking out locks that produce lock ordering inversions, and therefore deadlocks.

Worse, the code could resume on another thread (in advanced scenarios; normally you pick up again on the thread that did the await, but not necessarily) in which case the unlock would be unlocking a lock on a different thread than the thread that took out the lock. Is that a good idea? No.

I note that it is also a "worst practice" to do a yield return inside a lock, for the same reason. It is legal to do so, but I wish we had made it illegal. We're not going to make the same mistake for "await".

Use SemaphoreSlim.WaitAsync method.

 await mySemaphoreSlim.WaitAsync();
 try {
     await Stuff();
 } finally {
     mySemaphoreSlim.Release();
 }

Basically it would be the wrong thing to do.

There are two ways this could be implemented:

• Keep hold of the lock, only releasing it at the end of the block.
  This is a really bad idea as you don't know how long the asynchronous operation is going to take. You should only hold locks for minimal amounts of time. It's also potentially impossible, as a thread owns a lock, not a method - and you may not even execute the rest of the asynchronous method on the same thread (depending on the task scheduler).

• Release the lock in the await, and reacquire it when the await returns
  This violates the principle of least astonishment IMO, where the asynchronous method should behave as closely as possible like the equivalent synchronous code - unless you use Monitor.Wait in a lock block, you expect to own the lock for the duration of the block.

So basically there are two competing requirements here - you shouldn't be trying to do the first here, and if you want to take the second approach you can make the code much clearer by having two separated lock blocks separated by the await expression:

// Now it's clear where the locks will be acquired and released
lock (foo)
{
}
var result = await something;
lock (foo)
{
}

So by prohibiting you from awaiting in the lock block itself, the language is forcing you to think about what you really want to do, and making that choice clearer in the code that you write.