KeyVaultErrorException: Operation returned an invalid status code 'Forbidden'

It is dodgy, but the problem with your setup is that you enabled "Authorized application" when you created the Access policy for your KeyValut.

You can see it because you have "Application+Application" on your screenshot. I imagine, you created the policy with your webapp being both principal and authrorized application. This kinda creates a loop.

To make this setup work just delete your existing policy and create a new one, where you select only the Principal:

only select principal and add web app,don't select authorized application

For me to resolve the issue, I had to add the IP addresses of the web app to the firewall as not all of them were getting added when you select "Allow trusted Microsoft services..."

Thanks to: https://azidentity.azurewebsites.net/post/2019/01/03/key-vault-firewall-access-by-azure-app-services