What exactly is 'UseAuthentication()' for?

You do need to call it.

UseAuthentication() is documented as:

https://docs.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.builder.authappbuilderextensions.useauthentication?view=aspnetcore-2.0

Adds the AuthenticationMiddleware to the specified IApplicationBuilder, which enables authentication capabilities.

It basically does this:

IApplicationBuilder AddAuthentication(this IApplicationBuilder app) {

    return app.UseMiddleware<AuthenticationMiddleware>();
}

...so it's just saving you some typing and possibly some extra using imports.

This adds an AuthenticationMiddleware instance into the process' request-handling pipeline, and this particular object adds the plumbing for authentication.

Although this is an old thread, but since I stumbled with the same question recently I thought shedding some more light on the internals may benefit others

The short answer is depends on your service type and your APIs. you don't need to call UseAuthentication when:

1. You implement your own middleware that handles authentication - no need to elaborate here. You handle everything yourself and obviously don't need additional dependencies
2. You don't need automatic or remote authentication

Remote authentication

Authentication that requires redirect to identity provider, like OpenID Connect.

What makes is so special?

These middlewares need to correlate different http calls.

An initial calls is first processed by the middleware, then redirected to the identity provider (where the user needs to login) and then back to the middleware. In this case the middleware needs to own the request and not allow other authentication middlewares to participate in the process.

This is first part of the middleware code:

// Give any IAuthenticationRequestHandler schemes a chance to handle the request
var handlers = context.RequestServices.GetRequiredService<IAuthenticationHandlerProvider>();
foreach (var scheme in await Schemes.GetRequestHandlerSchemesAsync())
{
    var handler = await handlers.GetHandlerAsync(context, scheme.Name) as 
    IAuthenticationRequestHandler;
    if (handler != null && await handler.HandleRequestAsync()) 
    {
        return;
    }
}

• This is of course a simplified explanation, as remote handlers are more complicated. the goals is eventually to focus and explain the behavior of the middleware

Automatic authentication

Authentication that runs automtically for the default scheme. As the name suggest, if you have defined a default authentication scheme, then authentication handler that is associated to th middleware will always run.

Intuitively you would expect authentication middlewares to run first, specifically they should run before the MVC layer (i.e. Controllers). But, this also this means that the authentication layer doesn't know which controllers should run or about the authorization requirements of those controllers, in other words it doesn't know what is the authorization policy [Authorize("Policy")] it should evaluate.

So logically, we would like to first evaluate the policy and only then run the authentication logic. This is the reason why authentication handlers move in ASP 2.* to be general services and not coupled to middlewares.

But, in some cases you always want the authentication handler to run, regardless of your policy. In this case you can define default authentication schemes which will automatically run.

This explains the second part of the middleware code:

var defaultAuthenticate = await Schemes.GetDefaultAuthenticateSchemeAsync();
if (defaultAuthenticate != null)
{
    var result = await context.AuthenticateAsync(defaultAuthenticate.Name);
    if (result?.Principal != null)
    {
        context.User = result.Principal;
    }
}

If you are developing a REST API that supports multiple authentication schemes or have mixture of authenticated and non authenticated controllers, then you don't need automatic authentication since it adds redundancy.

Conclusion

That brings us to the interesting question and the answer: when and where does authentication occur when its not automatic and not remote?

In the normal MVC authorization flow, this happens in the AuthorizeFilter class that is calling IAuthenticationService.AuthenticateAsync

• You can call this method yourself if you implement your own authorization layer or when working with lower level APIs (e.g. websockets which are not implemented as controllers)

For these cases, calling UseAuthentication is not required

If you write your custom middleware (like you do in your example), you don't need to call AddAuthentication because the authentication middleware won't be aware of your own.

That being said, you probably don't want to create your own middleware: you probably want to create a new authentication handler that plays nicely with the ASP.NET authentication framework (so that you use the [Authorize] attribute on controllers).

To create a custom authentication, you have to create a dedicated handler that inherit from AuthenticationHandler, and implements the relevant methods. You can have a look at an example of basic authentication on github: https://github.com/blowdart/idunno.Authentication, but here's a quick example to show the gist of the custom handlers.

public class BasicAuthenticationOptions : AuthenticationSchemeOptions
{
    public BasicAuthenticationOptions()
    {
    }
}

internal class BasicAuthenticationHandler : AuthenticationHandler<BasicAuthenticationOptions>
{
    private const string _Scheme = "MyScheme";

    public BasicAuthenticationHandler(
        IOptionsMonitor<BasicAuthenticationOptions> options,
        ILoggerFactory logger,
        UrlEncoder encoder,
        ISystemClock clock) : base(options, logger, encoder, clock)
    {
    }

    protected override async Task<AuthenticateResult> HandleAuthenticateAsync()
    {
        string authorizationHeader = Request.Headers["Custom-Auth-Handler"];

        // create a ClaimsPrincipal from your header
        var claims = new[]
        {
            new Claim(ClaimTypes.NameIdentifier, "My Name")
        };

        var claimsPrincipal = new ClaimsPrincipal(new ClaimsIdentity(claims, Scheme.Name));
        var ticket = new AuthenticationTicket(claimsPrincipal,
            new AuthenticationProperties { IsPersistent = false },
            Scheme.Name
        );
        
        return AuthenticateResult.Success(ticket);
    }
}

You can then register your new scheme in Startup.cs:

public void ConfigureServices(IServiceCollection services)
{
    services
        .AddAuthentication(BasicAuthenticationDefaults.AuthenticationScheme)
        .AddScheme<BasicAuthenticationOptions, BasicAuthenticationHandler>("MyScheme", options => { /* configure options */ })
}