Azure Key Vault vs Azure Environment Variables - Which is the right way?

However in the case of a simple web app I don't see the benefit of using environment variables which can be configured in the portal.

I have not used Azure Key Vault, but it appears that the client application never actually sees the master key:

When a SaaS application needs to perform cryptographic operations by using their customers’ keys, Key Vault does this on behalf of the application. The application does not see the customers’ keys.

If the application is compromised then in both scenarios the attacker can decrypt whatever they want. However, if the application (and therefore the attacker) never has direct access to the keys then once the attacker loses access they can no longer decrypt any further data. This protects future data, and data which the attacker wasn't able to decrypt before losing access from being compromised.

If the master key was simply in an environment variable and the attacker compromised it, then even if you completely revoked their access they could continue to decrypt data they originally stole.

An additional benefit is that presumably every decryption operation is logged, so you can more easily audit the scope of the data which was compromised. However, if the attacker had direct access to the key then they could decrypt it offline and you'd never know what was and wasn't compromised.

Azure Key Vault does not only offer crypto solutions. It is also a user-based secret store. This solves one problem of secret management, which is quick remediation. Imagine that you have a SQL database that has to be accessed by multiple hosts, let's say hosts A and B. If the database password is compromised (typically because one host was attacked) and you put them in environment variables at both hosts, you have to heal both hosts.

With a user-based secret store, A and B have distinct credentials. If A is attacked, the attacker may be able to retrieve the SQL database password, but you can immediately revoke the authorization of A from the Key Vault, change the password at SQL database, and update the Key Vault with the new password. Note that no action was performed on B. When B identifies itself to the secret store, it will retrieve the updated SQL password and resume operation.

Of course, if your database password is used by single web app, then this doesn't help much. Still, you may have crypto applications that benefit from never seeing the private key, as the other answer explained.