How to delete (invalid) files with colon in their name under Windows?

You may want to try booting from an Ubuntu Live CD and deleting it from there.

You may be able to use this syntax (per Microsoft KB 320081):

del "\\?\c:\path_to_file\bad:name.ext"

See also this serverfault question on the same topic, "delete file containing invalid characters in windows".

I think the old "ntfsdos" sysinternals utilities could remove/rename these files as well, but these are no longer available post-MS acquisition.

Close all the files, if any opened, that are saved in the drive which is containing such files. Now, Open the drive properties.

"My Computer" -> "Right Click on the this drive" > properties

Then, go to "Tools" tab and click "Check now" under 'Error checking' label.

This will unmount your drive and will scan the complete drive for errors, eventually it will delete such files with illegal names. On finishing it will show a messages saying some files were fixed.

I successfully deleted such files from my external hard disk.