Does adding two-factor authentication by OTP really make KeePass more secure
No. Security remains the same + extra cognitive overhead.
Presumably the plugin uses OATH HOTP where the KeePass file or master key is re-encypted after each access with the next one-time-password.
However to generate the next password on the device, the plugin would require either a secret stored on the device or the normal password for the KeePass file.
Security from a one-time-password comes from two parties knowing the same key and counter - HOTP(Key,Counter) - while an attacker doesn't know the key. If the attacker has access the device storing the KeePass installation and files, the security re-collapses to the security of the normal password on its own. If the KeePass file is still interoperable with other KeePass programs, then you gain nothing from using a one-time-password in this fashion.
One-time-passwords work well for server authentication because both client and server end-points are considered secure and the attacker needs 'something you own' as well as 'something you know'. If the attacker has your computer, then they now have 'the thing you own'.
From reading the source it looks like the secret is stored on the Yubikey and stored encrypted multiple times with different keys inside the otp.xml file. The encryption keys are derived from the next n OTPs (derived from the secret) starting from OTP i..i+m where i is the current counter and m is the look ahead value. When the user enters their OTP values, the plugin creates a key from those values and tries to decrypt one of the encrypted copies of the secret using that key. If successful, the secret is used to derive the next n+m OTPs and generate the m keys used to encrypt the secret for next time before unlocking the database. This means that the protection cannot be bypassed without having some copy of the secret, which is not stored in plain text alongside the database.
Although apparently secure, I don't like this plugin because of the possibility for the counters to go out of sync. Be sure you have a copy of the secret written down somewhere safe.
KeePass has a master-key that is encrypted with your password normally. If that key is encrypted with something generated by the OTP and your password, then both mechanisms would be needed to decrypt the master key that encrypts the data store. That said, I'm not sure how they may have implemented a OTP setup that is secure since normally a OTP is a method of authentication rather than key storage (ie, I don't know how you would have the OTP unlock the key).
It is also suspect that they could effectively prevent replay since the db is controlled by an attacker in most attack scenarios, just not necessarily in the way you expected. They may have some way around this problem, but I can't think of one.