What is the risk of having a 2FA key permanently plugged into my device?

The threat model for the Nano is protecting accounts from remote access, not from direct access from an approved device. You essentially make the device itself the "thing you have" factor with the benefit that the "thing's" properties cannot be stolen remotely (as is the case for private keys, cookies, etc.).

Convenient? Yes. Easy to add to your grandmother's laptop and everyone to forget about while still maintaining protection? Yes. Easy to lose? No. Are there "more secure" methods? Yes.

You can now sign into Microsoft 365 just using a FIDO2 Compatible device. You do not need to enter a password.

Microsoft 365 does make you assign a pin to unlock your device. So if someone steals it, they cannot use it to get into your 365 account without the pin. Even if they have you machine with your YubiKey 5 Nano in it, they still can't get into your 365 account without knowing the pin.

This method is far more secure than most other sign in methods.