What are #file# and file~ and how can I get rid of them?

I recommend installing no-littering. It automatically puts backup files (file~) in ~/.emacs.d/var/backup/. It doesn't do anything about autosaves (#file#), but there is a note about putting those files in a specified directory in the README:

(setq auto-save-file-name-transforms
      `((".*" ,(no-littering-expand-var-file-name "auto-save/") t)))

Neither of these things actually prevents Emacs from creating these files, but I'm assuming most people actually want these files (in case of a crash), but don't want them strewn all over the filesystem.

For #files# you have to do rm "#file#" from the terminal, because rm #file# doesn't work. For ~file you can simply digit rm ~file.

The file with the ~ is a backup file that automatically gets created when you save a file. The #readme.txt# is the file being currently edited/in use (i.e., the autosave version). That will usually go away (unlike the ~ file) when you exit emacs normally (if it crashes or gets killed the # files may stay around).

You might find this page about emacs backup files of interest, and this SO question: How do I control how Emacs makes backup files?

You can prevent backup files from being created with this:

(setq make-backup-files nil)