What's the point of Microsoft Account 2FA if it still lets me log in using password instead?

You didn't actually set up 2FA. You set up your authenticator as an alternative method of single-factor authentication. This is clear from the first screenshot: "... to sign in without a password". If it didn't ask you for a password in the first place, it's probably not 2FA; the password is one of the two factors. The way I read this question it seemed like you'd gotten that prompt after entering your password, because that's when any second-factor authentication prompt would appear, but it looks like that's not what happened.

Go to https://account.live.com/proofs/manage/additional and click "Set up two-step authentication" if you actually want 2FA. You will still be able to "remember" trusted devices after you've completed the two-step auth on them, but any time you try to sign in using a new device (or a private browser, etc.) it should ask for both factors.

I can't be sure this is what is happening, but some implementations have the concept of a "trusted device" from where they only ask for 2FA once, then consider (at your request) that the device is fully under your control and you don't want to be bothered having to 2FA all the time.

Failing that, you're absolutely right, it completely negates the purpose of having 2FA.

Maybe it's configurable?