Does a password manager's master password significantly improve security?

All depends on what threats you consider. You are right, if your system is deadly compromised with keyloggers everywhere, there is little that can be done. But what if the attacker can just take a copy of a file and if that file is the password vault?

The common usage of a password manager like Keypass, is the assumption that there is a hierarchy in data sensitivity:

• not so sensitive data can live in unencrypted files on the disk
• highly sensitive data (passwords to bank accounts, PIN code of your credit card, etc.) should be protected with a master password in case of data theft.

If your data does not respect that hierarchy, it is possible that the security of the password vault is not worth a complex master password. But that means that you use Keypass only for its ability to store passwords and no longer for its ability to protect passwords. In that case, you could as well store passwords directly in your browser...