How to log unhandled exception in .net core

Middleware could be a way to go yes. I've written a guide here: Error Logging Middleware in ASP.NET Core.

I would probably look at Microsoft.Extensions.Logging in combination with a logging framework like Serilog, NLog or log4net.