How do I make the install4j updater remove files?

You can add an "Uninstall previous installation" action to your "Installation" screen.

If there are some files that you do not want to delete during such an update, go to the "Files" step, edit an entry, switch to the "Installation options" step of the wizard and set the "Uninstall policy" to one of the "... but not for update" options.