Why is it possible to convert a file just by renaming its extension?

This is what probably happens. The application designated to open files with the file types you mention, is the same across all of those. .gif, .jpg, .png: these are all handled by EOG (GNOME's Eye Of Gnome). I suspect this application does not determine the file type based on the extension, rather it seeks for the magic number.

Note that the behavior of EOG may very well be exceptional, and also beware that no actual conversion is done. It's just that Linux applications tend to rely on different detection methods than file extensions, though many modern desktops actually look for the file extension to determine what application to open, since this method is probably quicker.

This all may apply to Ubuntu, Windows is a completely different story. Explorer practices a behavior by default that cuts off the file extension from the filename. This way, a file named "document.pdf" will show as "document", and renaming it to "document.dvi" will actually be processed as "document.dvi.pdf". Viruses sometimes exploit this "feature" by making the user believe they are opening a .jpg, while in reality they are being tricked into running a malicious exe!

To be really sure about filetypes, there is a command called file that shows you detailed file information.

From my personal experience, I can tell that, at least on my system, EOG cannot be tricked into recognizing a renamed image file. You must be doing something different than I do.

Renaming .png files to .jpeg and .gif works because the program associated with and assigned to open those files is the same, and when it sees the extensions it can open, it simply opens them, because those are all image files and the program can open them.

The default program is EOG (Eye of Gnome) as another answerer says. Even if you remove the extension, it can still open them.

As for the dvi and pdf, again both file types are opened by the same program, which is called evince. So, the same thing happens there.