Aren't keyfiles defeating the purpose of encryption?

If decryption only relies on the keyfile and this keyfile is readily available, there is indeed no significant security benefit in your setup.

What you can do though is store the keyfile on a removable device (e.g. a USB stick) and detach it when you are not around. That way decryption is only possible when you are present and the removable device is attached.

Storing the keyfile locally makes sense if you want to ensure that a removable device can only be decrypted on your system. You can distribute the keyfile to other systems as well if you want to use the encrypted device in different places. If you lose the removable device in transit, little harm is done, because it can only be decrypted on a system that has your keyfile.

While Demento's answer is fine, I'd also present another user case I'm personally using: I have full disk encryption set up — using LUKS on a Linux-based box, and the scheme is implemented as follows:

• The boot partition is encrypted and requires a password to be entered so the boot manager (GRUB) is able to mount it.
• The partition contains the kernel image and the so-called "initramfs" which is an "early-boot userspace" used to bring up the rest of the system. This image contains the unencrypted keys used to decrypt other attached LUKS-encrypted partitions.
  The image is kept with as tight permissions as possible.

The upside of this setup is that it requires the user to only enter their passphrase once.
The obvious downside is that if the attacker somehow manages to decrypt the boot partition they automatically gain access to the rest of the disk space; if an attacker somehow manages to mount a local root exploit on a running system to gain access to the initramfs image available on the decrypted partition exported by the LUKS subsystem, it gains access to the partitions other than the boot (though, in the case of such an exploit they'd get such access right away as the partitions are already decrypted and exported).

Here, the usage of the key to encrypt data storage is to replace the need to provide a passphrase (given the key itself is kept in a reasonably secure manner).