Named Credentials and support for the OAuth2 Client Credentials Grant Type and alternatives
In my experience, the bullet points you've highlighted are pretty much on-target. What I would add is that both NamedCredentials and CustomApplications, make the assumption the OAuth provider (the idP) has a publicly accessible Social or other major API along the lines of Facebook, Google, Twitter, etc where you can import a token or some other credential to begin with that identifies the provider. If it doesn't have that capability, it doesn't seem to want to support allowing you to do simple OAuth authentication (as in "handshaking") using protocols with some other endpoint. In essence, if you aren't doing SSO using for Salesforce Login, the Auth Namespace doesn't provide much assistance for use in OAuth custom flows that integration partners may want you to use as part of their API.
Like you, I recently found this very frustrating. See the following question Unusual OAuth JWT Bearer Token Flow (Outbound) Integration Pattern. I wound up having to write my own service to handle an OAuth Username-PW login to obtain a JWT Bearer Token, plus Refresh Token that expired after 12 hours. I couldn't find a way to make named credentials work for me and no adequate documentation to describe how to extend the AuthProviderPlugin in a manner that would allow me to do this I all could fin is Create a Custom Authentication Provider Plug-in which isn't very helpful for this type of use case (again, it assumes SSO for Salesforce login).
In examining the AuthProviderPlugin, and the Auth Namespace in general, what I've concluded is that it is almost entirely written around support for login to Salesforce and negotiation for OAuth where the idP only supports login to Salesforce; not login to other services. That generally seems to be its purpose. It is not written for OAuth negotiations for transactions that take place in the opposite direction.
Two libraries I discovered which may be helpful to you are ffhttp-core and ffhttp-core-samples. Both still expect an external provider which can include Salesforce as the idP, but are written for connecting to external services like Box (the example application illustrated in ffhttp-core-samples), GoogleDrive, etc and can be customized for any OAuth flow that you need to use. I chose not to implement ffhttp-core for my purposes, but found it helpful in figuring out what I needed to do in order to write my code.
Bit of a hack indeed, but there is a way to use Named Credentials to implement the Client Credentials flow so you don't have to manage access or refresh tokens yourself. All credit for figuring this out should go to Chuck Liddell, who did a brilliant Pluralsight course on this topic that I highly recommend (most relevant part to Client Credentials flow starts here: https://app.pluralsight.com/course-player?clipId=fbc87f2d-7675-418d-bba9-920248983ff7)
Step 1. Create a Custom Metadata Type (Name: Client Credentials Auth Provider Definition, or similar):
- Create 3 custom text fields on it:
- Client Id
- Client Secret (WARNING: this will be visible in plain text in the Setup menu)
- Token Endpoint
- You don't need to create any Custom Metadata records for this type
- This will define what fields are available to fill in later on your custom Auth Provider
Step 2. Create an Apex class (Name: {YourThirdPartyService}ClientCredential, or similar) that extends 'Auth.AuthProviderPluginClass' (or copy Chuck's class from here and tweak it: https://github.com/grekker/pbp-authenticating-external-apps/blob/master/force-app/main/default/classes/CustomAuthProvider.cls)
- Make sure your getCustomMetadataType method references the API name of the Custom Metadata Type you created in Step 1
- The name of the Apex class will now appear in the list of options for 'Provider Type' when creating a new Auth Provider
Step 3. Set up a custom Auth. Provider with:
- Provider Type: <Apex class name from Step 2>
- Client Id from your third party service
- Client Secret from your third party service
- Token Endpoint from your third party service
- Update the Apex class' redirectUrl variable to end with the URL Suffix defined in your new Auth Provider
Step 4. Set up a Named Credential with:
- URL: <your third party service's resource endpoint baseURL>
- Identity Type: Named Principal
- Authentication Protocol: OAuth 2.0
- Authentication Provider: <Auth Provider from Step 3>
- Start Authentication Flow on Save = checked
- Generate Authorization Header = checked
Upon saving, the Named Credential should get the access token back and store it securely. From this point forward, callouts using this Named Credential should work the same as any other, more standard Named Credential