What's the purpose of AsyncContext.start(...) in Servlet 3.0?

You found a poor example, IMHO. In fact I wasn't even aware of AsyncContext.start() existence.

I had a quick look at how Jetty and Tomcat implement this. In fact, they seem to have some thread pool that handles asynchronous invocations independently.

Such usage of the API gives you nothing or very little. Instead of blocking the HTTP thread you are blocking some other thread pool. So I can imagine the application still accepts new connections, but the problem remains - the container can't handle them all because that extra thread pool is still limited.

The whole points of AsyncContext is the ability to handle more than one request by a single thread. Often you need only a single thread to handle thousands of asynchronous connections - e.g. when exactly one thread waits for data that is suppose to be broadcasted to several clients. Also see The Limited Usefulness of AsyncContext.start()

I had the same reaction at first -- if you're just passing the work to another thread, what do you gain? The spec is not much help in explaining why this is a good idea. But this post does an excellent job. Basically, it's to let the server degrade gracefully under heavy load rather than just fail by running out of threads. The actual work is done in a fixed-size pool of threads, so the server can accept any number of requests without having to keep a thread around for each one until it completes. Of course, you may have to tweak your O/S settings to be able to keep open thousands of sockets at a time.

Once you have this ability, you can more easily take advantage of the Comet (server push) architecture, where the client Javascript keeps an AJAX request open so that the server can notify it as soon as some event occurs, rather than having to poll the server to find out if something happened.