Azure active directory application registration for localhost and production

Firstly, you can make it work easily without changing sign-on URL every time. You just need to make sure that both URLs.. i.e. https://localhost:44302/ as well as https://mycoolapp.com are available in the Reply URLs for this application.

The value for Sign-on URL that you give at the time of creating app registration goes into two places. It becomes the Home Page URL under Properties for your web application and it also gets added as the first URL in the Reply URLs collection for your web application. Just make sure to add a the second URL to the Reply URLs collection after your app is registered and that's all.

On a side note, even though you can make things work with a single app registration for both production and localhost debugging, you may want to think about having two separate application registrations for production v/s local development and give permissions to production application only for administrators/dev ops if it makes sense.

Imagine any sensitive settings, like if you are using a secret key for your web application, you may not want your development team to have access to secret key for your production app registration even while debugging code. You can always change which app registration gets used in dev v/s production code based on configurations.

It's some of the headaches like these where Managed Service Identities are very appropriate, but whether that makes sense or not is a little different discussion than this one.