Spring Security Filter chain executed twice per request. Why?

My controller endpoints respond asynchronously (Callable, DeferredResult). Apparently Spring passes the async processing thread for the whole Spring Security chain. Since in my case that's not necessary simply using the following configuration the chain is not invoked in the async thread anymore:

spring:
  security:
    filter:
      dispatcher-types:
        - request
        - error