How to set Ruby's load path externally

See the "Ruby and Its World" chapter from The Pickaxe Book, specifically the section on environment variables. Excerpt:

RUBYLIB 
  Additional search path for Ruby programs ($SAFE must be 0).
DLN_LIBRARY_PATH
  Search path for dynamically loaded modules.
RUBYLIB_PREFIX
  (Windows only) Mangle the RUBYLIB search path by adding this
  prefix to each component.

Make sure that you've placed the installed bin directory in your $PATH for the gem command to work. It should modify the RUBYLIB itself, but if not, try Martin's answer to fix that.

Then, you can have your gem home (where the gems that rubygems installs are stored) be local.

Just use $GEM_HOME (or set things up in your ~/.gemrc) and check that everything took with gem environment.

% mkdir ~/.gems
% export GEM_HOME=~/.gems
% gem help environment

Usage: gem environment [arg] [options]

  Common Options:
    -h, --help                       Get help on this command
    -V, --[no-]verbose               Set the verbose level of output
    -q, --quiet                      Silence commands
        --config-file FILE           Use this config file instead of default
        --backtrace                  Show stack backtrace on errors
        --debug                      Turn on Ruby debugging


  Arguments:
    packageversion  display the package version
    gemdir          display the path where gems are installed
    gempath         display path used to search for gems
    version         display the gem format version
    remotesources   display the remote gem servers
           display everything

  Summary:
    Display information about the RubyGems environment

  Description:
    The RubyGems environment can be controlled through command line arguments,
    gemrc files, environment variables and built-in defaults.

    Command line argument defaults and some RubyGems defaults can be set in
    ~/.gemrc file for individual users and a /etc/gemrc for all users.  A gemrc
    is a YAML file with the following YAML keys:

      :sources: A YAML array of remote gem repositories to install gems from
      :verbose: Verbosity of the gem command.  false, true, and :really are the
                levels
      :update_sources: Enable/disable automatic updating of repository metadata
      :backtrace: Print backtrace when RubyGems encounters an error
      :bulk_threshold: Switch to a bulk update when this many sources are out of
                       date (legacy setting)
      :gempath: The paths in which to look for gems
      gem_command: A string containing arguments for the specified gem command

    Example:

      :verbose: false
      install: --no-wrappers
      update: --no-wrappers

    RubyGems' default local repository can be overriden with the GEM_PATH and
    GEM_HOME environment variables.  GEM_HOME sets the default repository to
    install into.  GEM_PATH allows multiple local repositories to be searched
    for
    gems.

    If you are behind a proxy server, RubyGems uses the HTTP_PROXY,
    HTTP_PROXY_USER and HTTP_PROXY_PASS environment variables to discover the
    proxy server.

    If you are packaging RubyGems all of RubyGems' defaults are in
    lib/rubygems/defaults.rb.  You may override these in
    lib/rubygems/defaults/operating_system.rb