What is an effective method for installing up-to-date software on an out-dated production machine?

Your management is wise in not trying to upgrade a working cluster that is performing an important function based on a proprietary package.

Backporting packages is time consuming and risky, that is, not always feasible. You might avoid the time penalty if you can finding the packages that you want to install in the original CentOS 5.4 repository or in some CentOS 5.4 backport repository. While you can have several versions of GCC on one host at the same time (the embedded systems/cross compile folks do this all the time), but it is not trivial to have more than one glibc in a single run-time environment.

So, you are best advised to work in a separate, newer environment that has the packages that you need and find some way to test the output of the old environment in the new one. In any event, do not risk breaking anything in the old environment or you may need all of the stackexchange.com reputation points that you can get to find your next job ;-)

Installing packages from distributions is often difficult when you don't have root permissions, as they assume a fixed directory layout and the dependency system tends to require some packages with setuid or setgid programs that you can't install as non-root.

Compiling from source is more often than not the easiest way. (And if you're after speed, you can choose the best compilation options for your particular processor model.)

To organize the packages that you compile (or install by extracting tarballs), I recommend using stow or the more powerful but more complex xstow. Their basic mode of operation is to install each package in a separate directory, then create symbolic links to put them all together. Here's a typical compilation and installation session with stow:

tar xzf foobar-42.tar.gz
cd foobar-42
./configure --prefix=~/software/stow/foobar-42
make
make install
cd ~/software/stow
stow foobar-42

That last command creates symbolic links from files and directories under ~/software/stow under ~/software. For example, if ~/software/stow/foobar-42 contains a lib/foobar directory and files bin/foobar and man/man1/foobar.1, then you will end up with symbolic links

~/software/bin/foobar -> ../stow/foobar-42/bin/foobar
~/software/lib/foobar -> ../stow/foobar-42/lib/foobar
~/software/man/man1/foobar.1 -> ../../stow/foobar-42/man/man1/foobar.1

To uninstall a program, run stow -D foobar-42 in the ~/software/stow directory, and delete ~/software/stow/foobar-42. To make a program temporarily unavailable (e.g. to try another version), just run the stow -D part.

See also Non-Root Package Managers ; Best way to upgrade vim/gvim to 7.3 in Ubuntu 10.04?