What is the best way to uninstall gems from a rails3 project?

With newer versions of bundler you can use the clean task:

$ bundle help clean
Usage:
    bundle clean

Options:
    [--dry-run=only print out changes, do not actually clean gems]
    [--force=forces clean even if --path is not set]
    [--no-color=Disable colorization in output]
    -V, [--verbose=Enable verbose output mode]

Cleans up unused gems in your bundler directory
$ bundle clean --dry-run --force
Would have removed actionmailer (3.1.12)
Would have removed actionmailer (3.2.0.rc2)
Would have removed actionpack (3.1.12)
Would have removed actionpack (3.2.0.rc2)
Would have removed activemodel (3.1.12)
...

edit:

This is not recommended if you're using a global gemset (i.e. - all of your projects keep their gems in the same place). There're few ways to keep each project's gems separate, though:

1. rvm gemsets (http://rvm.io/gemsets/basics)
2. bundle install with any of the following options: --deployment or --path=<path> (http://bundler.io/v1.3/man/bundle-install.1.html)

Bundler now has a bundle remove GEM_NAME command (since v1.17.0, 25 October 2018).

Bundler is launched from your app's root directory so it makes sure all needed gems are present to get your app working.If for some reason you no longer need a gem you'll have to run the

    gem uninstall gem_name 

as you stated above.So every time you run bundler it'll recheck dependencies

EDIT - 24.12.2014

I see that people keep coming to this question I decided to add a little something. The answer I gave was for the case when you maintain your gems global. Consider using a gem manager such as rbenv or rvm to keep sets of gems scoped to specific projects.

This means that no gems will be installed at a global level and therefore when you remove one from your project's Gemfile and rerun bundle then it, obviously, won't be loaded in your project. Then, you can run bundle clean (with the project dir) and it will remove from the system all those gems that were once installed from your Gemfile (in the same dir) but at this given time are no longer listed there.... long story short - it removes unused gems.

This will uninstall a gem installed by bundler:

bundle exec gem uninstall GEM_NAME

Note that this throws

ERROR: While executing gem ... (NoMethodError) undefined method `delete' for #<Bundler::SpecSet:0x00000101142268>

but the gem is actually removed. Next time you run bundle install the gem will be reinstalled.