Is it possible to prevent players from using hacks on a vanilla server?

The vanilla server is exactly what it is and has no features to prevent these wallhacks and exploits. Your only tools in vanilla are player bans and kicks after people misbehave, or using the whitelist.

To automatically prevent or even detect these exploits you need to be running a modified server. There are many, many anti-cheat plugins for Bukkit, for example.

Absolutely! Lots of vanilla servers use command-block powered anticheat systems.

One I'm using is: http://www.reddit.com/r/Minecraft/comments/29wkwb/vanilla_anticheat_and_antifly_system/

Hope this helps :)