VSTS Build Pipeline: Test fails connecting to Azure Key Vault

Use the Azure CLI pipeline task to run integration tests that need KeyVault secrets successfully, without exposing any secrets in source control:

1. Create a Service Principal service connection in your Azure DevOps project.

2. Give the principal Get and List permissions to the Vault in Azure.

3. Run your integration tests inside an Azure CLI task:

   - task: AzureCLI@1
     inputs:
       azureSubscription: 'Your Service Connection Name'
       scriptLocation: 'inlineScript'
       inlineScript: 'dotnet test --configuration $(buildConfiguration) --logger trx'
   

   This works because the tests will run in the context of azure cli, which is where AzureServiceTokenProvider tries fetching a token from before it fails. Azure CLI handles the authentication and cleans up when the task is done.

You should not do the integration test of authentication to Azure KeyVault within Azure DevOps Pipelines build, because you are using Azure DevOps default hosted agents.

By default, the Azure DevOps Pipelines are using basic default hosted agents, and these hosted agents are not accessible from your Azure subscription. These are not surprising, because these hosted agents are common agents for all common build needs, including build/compile, running unit tests, getting test coverages, and all of these tasks has no other additional features such as having ActiveDirectory, database, and other actual authentication/requests to other party such as authentication to any Azure Keyvault. Therefore these agents by default are not registered in your Azure subscription.

If you want to have successful integration tests for these special needs, you have to create your own agents for Azure DevOps Pipelines build and release. Therefore, there is no other way to force Azure DevOps default agent to run your KeyVault authentication tests, other than creating your own agents and configure your Azure DevOps to use your own agents.

To create your own agents, consult this documentation from Microsoft:

https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=vsts#install

UPDATE 29th October, 2018:

For more clarity, I also reply for your "Update 3" workaround. There is no guarantee that your workaround will work nicely when Microsoft updates the Azure DevOps' default hosted agent. Therefore I also need to add more point: it's not a good practice to have integration test that relies on other party beyond the realm of your Azure DevOps Pipelines build such as connecting to a database server or using external authentications (even on Azure KeyVault) within your CI, especially if you are using Microsoft's default hosted agents.

Not just it will be error-prone due to invalid authentication configuration, but there's no guarantee that the further updates on the default hosted-agents would guarantee your third-party logic test will work.