Does asynchronous model really give benefits in throughput against properly configured synchronous?

Everybody knows that asynchrony gives you "better throughput", "scalability", and more efficient in terms of resources consumption.

Scalability, yes. Throughput: it depends. Each asynchronous request is slower than the equivalent synchronous request, so you would only see a throughput benefit when scalability comes into play (i.e., there are more requests than threads available).

Does asynchronous code actually perform so much better comparing to the synchronous code with correctly configured thread pool?

Well, the catch there is "correctly configured thread pool". What you're assuming is that you can 1) predict your load, and 2) have a server big enough to handle it using one thread per request. For many (most?) real-world production scenarios, either or both of these are not true.

From my article on async ASP.NET:

Why not just increase the size of the thread pool [instead of using async]? The answer is twofold: Asynchronous code scales both further and faster than blocking thread pool threads.

First, asynchronous code scales further than synchronous code. With more realistic example code, the total scalability of ASP.NET servers (stress tested) showed a multiplicative increase. In other words, an asynchronous server could handle several times the number of continuous requests as a synchronous server (with both thread pools turned up to the maximum for that hardware). However, these experiments (not done by me) were done on a expected "realistic baseline" for average ASP.NET apps. I don't how the same results would carry over to a noop string return.

Second, asynchronous code scales faster than synchronous code. This one is pretty obvious; synchronous code scales fine up to the number of thread pool threads, but then can't scale faster than the thread injection rate. So you get that really slow response to a sudden heavy load, as shown in the beginning of your response time graph.

I think the work you've done is interesting; I am particularly surprised at the memory usage differences (or rather, lack of difference). I'd love to see you work this into a blog post. Recommendations:

• Use ASP.NET Core for your tests. The old ASP.NET had only a partially-asynchronous pipeline; ASP.NET Core would be necessary for a more "pure" comparison of sync vs async.
• Don't test locally; there are a lot of caveats when doing that. I'd recommend choosing a VM size (or single-instance Docker container or whatever) and testing in the cloud for repeatability.
• Also try stress testing in addition to load testing. Keep increasing load until the server is totally overwhelmed, and see how both the async and sync servers respond.

As a final reminder (also from my article):

Bear in mind that asynchronous code does not replace the thread pool. This isn’t thread pool or asynchronous code; it’s thread pool and asynchronous code. Asynchronous code allows your application to make optimum use of the thread pool. It takes the existing thread pool and turns it up to 11.