Why derive from ControllerBase vs Controller for ASP.NET Core Web API?

why it is necessary to derive from ControllerBase instead of Controller for a Web API controller.

It is not strictly necessary, just more to the point. The Controller class derives from ControllerBase and adds some members that are only needed to support Views.

Basically:

public abstract class Controller : ControllerBase
{
    public dynamic ViewBag { get; }
    public virtual ViewResult View(object model) { }
    // more View support stuff
}

When you write an API then ControllerBase matches your requirements better but both will work.

From the documentation (emphasis mine):

Don't create a web API controller by deriving from the Controller class. Controller derives from ControllerBase and adds support for views, so it's for handling web pages, not web API requests. There's an exception to this rule: if you plan to use the same controller for both views and web APIs, derive it from Controller.

I seem to remember that there was no ControllerBase in the first MVC iterations, it was inserted later. Hence the slightly odd naming/inheritance structure.

FROM https://docs.microsoft.com/en-us/aspnet/core/web-api/?view=aspnetcore-3.1

Don't create a web API controller by deriving from the Controller class. Controller derives from ControllerBase and adds support for views, so it's for handling web pages, not web API requests.