What's the difference between `/usr/share/emacs/site-lisp/` directory and `/usr/share/emacs23/site-lisp/` directory?

It's possible to have multiple versions of Emacs installed on the same machine. Only Emacs 23 would load files from /usr/share/emacs23/site-lisp/. All versions of Emacs would load files from /usr/share/emacs/site-lisp/.

Generally, you just use /usr/share/emacs/site-lisp/, unless there's some reason the code applies only to a particular version of Emacs.

To see where your Emacs is loading code from, type Control+h    v load-path Enter.