Azure key vault: access denied

To fix access denied you need to configure Active Directory permissions. Grant access to KeyVault.

1. Using PowerShell Run next command:

Set-AzureRmKeyVaultAccessPolicy -VaultName 'XXXXXXX' -ServicePrincipalName XXXXX -PermissionsToKeys decrypt,sign,get,unwrapKey

2. Using the Azure portal

1. Open Key Vaults
2. Select Access Policies from the Key Vault resource blade
3. Click the [+ Add Access Policy] button at the top of the blade
4. Click Select Principal to select the application you created earlier
5. From the Key permissions drop down, select "Decrypt", "Sign", "Get", "UnwrapKey" permissions
6. Save changes

Authorize the application to use the key or secret

The question did specify using the Azure Portal, I've documented creating a service principal for Key Vault access here.

Specifically from Step 2:

Open the Key Vault in the Azure Portal and select the Access policies blade under Settings. Click Add New and click on Select principal - you'll have to enter the full name of the registered app you created in the previous step in the search box before it'll show up, at which point you'll be able to select it.

You can either select an appropriate template from the top dropdown or choose Key, Secret or Certificate permissions manually. Don't worry about Authorized application at this stage.

IMPORTANT: pressing the OK button will add your new policy to the list, but it will not be saved! Be sure to click Save before continuing.

What is happening - your service principal doesn't have permissions to perform said operation. Take a look at this thread.

How do I fix an "Operation 'set' not allowed" error when creating an Azure KeyVault secret programmatically?