Removing sources after building from them

You don't need anything except the binaries. Those should already be copied to another location by make install. The rest can be removed. If you're unsure make a backup before removal. Remove, test if still working, if still working remove the backup.

Note: As ændrük pointed out: Using checkinstall over make install has the virtue of enabling Ubuntu to track the installation. Later you can revert any changes made using the regular Ubuntu package management.

If you installed files using make install, the only way to know where they went later on if you want to uninstall them is to inspect the source files. You may want to archive a copy of the source for this purpose.

A better alternative is to use checkinstall instead of make install. This allows Ubuntu's package manager to track each of the installed files in the same way it tracks software from the repositories. When using this method of installation, you don't have to keep any extra files around.

Generally once you got your binaries installed you can remove the directory containing source code, unless you want to compile them again, possibly with other compilation options.

Sometimes you could find particularly useful to keep the source directory, for example when upgrading Apache.

You can decide what's your best, in my opinion keep the source directory is a good thing. Obviously if you don't have disk space issues.