Uninstalling programs in Linux
A new install will seldom break your system (unless you do weird stuff like mixing source and binary).
If you use precompiled binaries in Ubuntu then you can remove them and not have to worry about breaking your system, because a binary should list what it requires to run and your package manager will list what programs rely on that program for you to review.
When you use source, you need to be more careful so you don't remove something critical (like glib). There are no warnings or anything else when you uninstall from source. This means you can completely break your machine.
If you want to uninstall using apt-get then you'll use apt-get remove package as previously stated. Any programs that rely on that package will be uninstalled as well and you'll have a chance to review them.
If you want to uninstall then generally the process is make uninstall. There is no warning (as I said above).
make config will not alter your system, but make install will.
As a beginner, I recommend using apt-get or whatever distro you use for binary packages. It keeps things nice and organized and unless you really want to it won't break your system.
Hopefully, that clears everything up.
In theory, make uninstall should remove what make install added and your system not accumulate cruft. Problem, of course, is that not all makefiles are created equal.
Some may omit the uninstall rule, leaving it to you to figure out what the install rule did. Worse, if install rule overwrote a linked library, dumb uninstall routine may break the dependencies for some other program.
Best solution for source installs is to use different prefix than the packages installed by the system's packaging manager. Apt installs files to /usr/ so use /usr/local/ hierarchy for your source installs. That makes it a lot easier to keep track of what files belong to which packages and uninstalls won't break the system.
./configure --prefix=/usr/local works for many configure scripts. If not, you can edit Makefile manually. Or just copy the files manually.
Apt and other packaging managers keep track of what files they've installed and their reverse dependencies so their uninstall functions are safe to use.
I would recommend you you use apt-get install to install any package in linux and apt-get remove (package name) or apt-get purge (package name) which will remove not only the main package that you are want to uninstall but all the associated packages or dependencies that were installed during the installation.
Now, to keep your system cleaner I'd recommend you to use apt-get clean https://askubuntu.com/questions/144222/how-do-apt-get-clean-and-apt-get-clean-all-differ#144224 (this post is interesting about that) which will remove all the files that were downloaded during installation but are not longer needed.
Another command that would be useful if you want to remove all the dependencies that are installed in your system but they weren't removed when you uninstalled is apt-get autoremove.
If you install a package via make and make install you'll be responsible for uninstalling it yourself (maybe there's a README file included in the downloaded package that tells you how to do it) as well as trying to uninstall all the dependencies associated with it. That's why is always recommended to install packages in Linux that are offered by the package manager of the distro, if you do it this way you can be sure your package has been tested enough to work with the distro (flavor of Linux) you are using and is very unlikely to break your system. Also, you can be sure your package will get updated when needed whereas if you install it by yourself you are the responsible to do all this.
I hope this helps :)