How to completely remove my Emacs?

You haven't installed Emacs with the package manager. As you have installed it from source tarball, try this way.

1. Check emacs version.

   $ emacs --version
   

2. Download the same emacs version you have installed in the past.

   $ wget https://ftp.gnu.org/gnu/emacs/emacs-$VERSION.tar.xz
   

3. Extract tarball.

   $ tar xJvf emacs-$VERSION.tar.xz
   

4. Run ./configure to generate the make file. If you remember the option you have given when you install it, add those command line flags, too. (e.g. --with-x=no)

   $ cd emacs-$VERSION
   $ ./configure
   

5. Do make uninstall to uninstall.

   $ sudo make uninstall
   

Enjoy!

apt-get can not detect the applications which you have installed from source.Actually every source file will have uninstall script also but unfortunately you have removed the source file.

You may still do this. In the terminal type as

locate emacs

it will list all it footprints in the system , then remove those emacs footprints by using rm command carefully.

It is not enough to

sudo apt-get remove emacs

You have to do

sudo apt-get remove emacs emacs23 emacs24

Then

locate emacs

keeps silent :-)