LD_LIBRARY_PATH doesn't seem to work

Try setting LIBRARY_PATH, instead of LD_LIBRARY_PATH.

From the gcc man page:

LIBRARY_PATH

The value of LIBRARY_PATH is a colon-separated list of directories, much like PATH. When configured as a native compiler, GCC tries the directories thus specified when searching for special linker files, if it can't find them using GCC_EXEC_PREFIX. Linking using GCC also uses these directories when searching for ordinary libraries for the -l option (but directories specified with -L come first).

Make sure that you export LD_LIBRARY_PATH after modifying it. Otherwise GCC won't be able to see the modified version.

LD_LIBRARY_PATH=/usr/lib/gcc/x86_64-redhat-linux/3.4.6:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH
gcc -o test test.c -lg2c