Can't remove keys from ssh-agent. Even rebooting doesn't help

It looks like it might be a bug. I am having similar behavoir in Ubuntu 10.10. A google search found a bug report for Debian: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=472477

To remove the extra keys I had showing, I just moved them out of the ~/.ssh directory.

Your keys are stored as files in the hidden directory: /home/jruser/.ssh/ that's how they persist after reboots. My guess the is that ssh-add -D is deleting them from memory but when you reboot they are read from the .ssh directory and so you have them again.