EF Core Connection to Azure SQL with Managed Identity

Is this a right way to do it or will it have issues with performance?

That is the right way. OnConfiguring is called for each new DbContext, so assuming you don't have any long-lived DbContext instances, this is the right pattern.

Do I need to worry about token expiration? I am not caching the token as of now.

AzureServiceTokenProvider takes care of caching.

Does EF Core has any better way to handle this?

The AAD Auth methods for SqlClient in .NET Core are documented here.

While the approach is generally correct in the sense that there is no other way than having to write custom code that sets the AccessToken of the connection, there is a couple of issues in your implementation that could be avoided by using a DbConnectionInterceptor as I will describe below. Those two issues are:

1. You took the responsibility of creating the connection object yourself. But you don't dispose it. Disposal will be tricky in your implementation, and that's why you might have skipped it.
2. Your code is blocking, as you use .Result to block while waiting for the access token.

A better alternative is to use interceptors, which EF Core supports. You will start with a DbContext like this:

public class MyCustomDbContextFactory : IMyCustomDbContextFactory
{
    private readonly string _connectionString;
    private readonly AzureAuthenticationInterceptor _azureAuthenticationInterceptor;
    public MyCustomDbContextFactory(DbContextFactoryOptions options, AzureAuthenticationInterceptor azureAuthenticationInterceptor)
    {
        _connectionString = options.ConnectionString;
        _azureAuthenticationInterceptor = azureAuthenticationInterceptor;
    }
    public MyCustomDbContext Create()
    {
        var optionsBuilder = new DbContextOptionsBuilder<MyCustomDbContext>();
        optionsBuilder
            .UseSqlServer(_connectionString)
            .AddInterceptors(_azureAuthenticationInterceptor);
        return new MyCustomDbContext(optionsBuilder.Options);
    }
}

And this is the interceptor implementation:

public class AzureAuthenticationInterceptor : DbConnectionInterceptor
{
    private const string AzureDatabaseResourceIdentifier = "https://database.windows.net";
    private readonly AzureServiceTokenProvider _azureServiceTokenProvider;
    public AzureAuthenticationInterceptor(AzureServiceTokenProvider azureServiceTokenProvider) : base()
    {
        _azureServiceTokenProvider = azureServiceTokenProvider;
    }
    public override async ValueTask<InterceptionResult> ConnectionOpeningAsync(DbConnection connection, ConnectionEventData eventData, InterceptionResult result, CancellationToken cancellationToken = default)
    {
        if (connection is SqlConnection sqlConnection)
        {
            sqlConnection.AccessToken = await GetAccessToken();
        }
        return result;
    }
    public override InterceptionResult ConnectionOpening(DbConnection connection, ConnectionEventData eventData, InterceptionResult result)
    {
        if (connection is SqlConnection sqlConnection)
        {
            sqlConnection.AccessToken = GetAccessToken().Result;
        }
        return result;
    }
    private Task<string> GetAccessToken() => _azureServiceTokenProvider.GetAccessTokenAsync(AzureDatabaseResourceIdentifier);
}

And this is how to configure your services:

services.AddSingleton(new DbContextFactoryOptions(connection_string));
services.AddSingleton(new AzureAuthenticationInterceptor(new AzureServiceTokenProvider()));

And finally, this is how to instantiate DbContext objects in your repository:

public async Task<IEnumerable<MyCustomEntity>> GetAll()
{
using var context = _notificationsDbContextFactory.Create();  // Injected in ctor
var dbos = await context.MyCustomEntity.ToListAsync();
return ... // something;
}