Remove test::unit

In your config/application.rb try

config.generators do |g|
  g.test_framework :rspec #=> or whatever
end

In response to comment

Try

config.generators do |g|
  g.test_framework nil
end

May not be the "most appropriate" or Rails-ish way, but it works

Update

I was reading the Rails Initialization Guide today and realized that the most likely reason Test::Unit is still being included is this line:

require 'rails/all'

Which could be replaced with:

require "active_record/railtie"
require "action_controller/railtie"
require "action_mailer/railtie"
require "active_resource/railtie"

This should take care of the issue. If you're using Rails 3.1.x you'd also include

require "sprockets/railtie"

if you're planning on using the asset pipeline.

Update 2

For Rails 3.2 you should use this:

config.app_generators do |c|
  c.test_framework :rspec, :fixture => true,
                           :fixture_replacement => nil

  c.integration_tool :rspec
  c.performance_tool :rspec
end

To remove Test:Unit in Rails 4:

Replace the following line from config/application.rb:

require 'rails/all'

With:

require "active_record/railtie"
require "action_controller/railtie"
require "action_mailer/railtie"
require "sprockets/railtie"

You can safely remove the test directory as well.

Build a new application with rails new APP_NAME --skip-test-unit in Rails 4 results in:

require "rails"
# Pick the frameworks you want:
require "active_model/railtie"
require "active_job/railtie"
require "active_record/railtie"
require "action_controller/railtie"
require "action_mailer/railtie"
require "action_view/railtie"
require "sprockets/railtie"
# require "rails/test_unit/railtie"