What exactly is the 'Saff Squeeze' method of finding a bug?
The sample shows that he is copying (inlining) the code under test inline into his unit test. Then testing the parts of the code separately from the begin to the end. This enables him to test each path in isolation and the produce unit test on the smallest possible units. One of the tests will demonstrate the defect and you will be able to fix your defect. The sample he shows is depending on the ability of Eclipse to inline methods, if you do not have this you need to do that by hand (copying the called code to your unit test).
The Saff Squeeze is a systematic technique for deleting both test code and non-test code from a failing test until the test and code are small enough to understand.
I agree that Kent's original description of the Saff Squeeze is a little difficult, partly because the software he's testing, JUnit, is highly abstracted, and partly because he doesn't give enough examples of step 2, "Place a (failing) assertion earlier in the test than the existing assertions."
In his first round he just moves the assertion higher in the test, and his summary of later steps might lead you to think that the only thing you can do in step 2 is move existing assertions, but by his final step he's come up with a new, simpler failing assertion. The assertion in step 2 can just be an existing one moved higher in the test, which is common, but it can also be a new one that you come up with as your understanding of the code and the bug evolves.
Here's an example. It's too simple to need the Saff Squeeze, but it illustrates the technique.
I just wrote this mission-critical class:
class Autopilot
def self.fly_to(city)
runways_in_city = runways_in city
runway = closest_available runways_in_city
flight_plan = flight_plan_to runway
carry_out flight_plan
end
def self.runways_in(city)
Airport.where(city: city).map(&:runways).flatten
end
def self.closest_available(runways)
runways.select { |r| r.available? }
.sort_by { |r| distance_between current_position, r.position }.last
end
def self.flight_plan_to(runway)
FlightPlan.new runway.latitude, runway.longitude
end
# other methods left to the imagination
end
Here's the first rspec example I wrote to test it:
describe Autopilot
describe ".fly_to" do
it "flies to the only available runway" do
Autopilot.stub(:current_position) { Position.new 0, 0 }
nearby_runway = create :runway, latitude: 1, longitude: 1
create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
# Think of the following line as being at the end of the example, since that's when it takes effect
Autopilot.should_receive(:carry_out).with flight_plan
Autopilot.fly_to nearby_runway.airport.city
end
end
end
Oh no -- the last line fails with this message: "Expectation failed: Expected Autopilot.carry_out to be called with FlightPlan(latitude: 1, longitude: 1), but it was called with FlightPlan(latitude: 2, longitude: 2)". I have no idea how that happened. We'd better use the Saff Squeeze.
Inline the method (renaming a local to avoid name collision):
it "flies to the only available runway" do
Autopilot.stub(:current_position) { Position.new 0, 0 }
nearby_runway = create :runway, latitude: 1, longitude: 1
create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
Autopilot.should_receive(:carry_out).with flight_plan
runways_in_city = runways_in city
runway = closest_available runways_in_city
actual_flight_plan = flight_plan_to runway
Autopilot.carry_out actual_flight_plan
end
I don't see how that last line could fail to meet the expectation, as long as it's getting the right FlightPlan
. Let's see if we can write a failing assertion higher up in the test:
it "flies to the only available runway" do
Autopilot.stub(:current_position) { Position.new 0, 0 }
nearby_runway = create :runway, latitude: 1, longitude: 1
create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
Autopilot.should_receive(:carry_out).with flight_plan
runways_in_city = runways_in city
runway = closest_available runways_in_city
actual_flight_plan = flight_plan_to runway
actual_flight_plan.should == flight_plan
Autopilot.carry_out actual_flight_plan
end
Ah, the new assertion fails too, with "expected FlightPlan(latitude: 1, longitude: 1), but got FlightPlan(latitude: 2, longitude: 2)". OK, let's simplify the test:
it "flies to the only available runway" do
Autopilot.stub(:current_position) { Position.new 0, 0 }
nearby_runway = create :runway, latitude: 1, longitude: 1
create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
runways_in_city = runways_in city
runway = closest_available runways_in_city
actual_flight_plan = flight_plan_to runway
actual_flight_plan.should == flight_plan
end
We're getting somewhere, but I still don't see what's wrong. Better Saff Squeeze again, inlining flight_plan_to
:
it "flies to the only available runway" do
Autopilot.stub(:current_position) { Position.new 0, 0 }
nearby_runway = create :runway, latitude: 1, longitude: 1
create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
runways_in_city = runways_in city
runway = closest_available runways_in_city
actual_flight_plan = FlightPlan.new runway.latitude, runway.longitude
actual_flight_plan.should == flight_plan
end
Well, obviously that's going to pass as long as flight_plan_to
gets the right Runway. Let's assert that:
it "flies to the only available runway" do
Autopilot.stub(:current_position) { Position.new 0, 0 }
nearby_runway = create :runway, latitude: 1, longitude: 1
create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude
runways_in_city = runways_in city
runway = closest_available runways_in_city
runway.should == nearby_runway
actual_flight_plan = FlightPlan.new runway.latitude, runway.longitude
actual_flight_plan.should == flight_plan
end
Good, the new assertion fails, with "expected Runway(id: 1) but got Runway(id: 2)". Simplify the test again:
it "flies to the only available runway" do
Autopilot.stub(:current_position) { Position.new 0, 0 }
nearby_runway = create :runway, latitude: 1, longitude: 1
create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
runways_in_city = runways_in city
runway = closest_available runways_in_city
runway.should == nearby_runway
end
We've pruned our original test and code to the point where it's obvious that the bug is in closest_available
-- it should use first
instead of last
.
But what if it's still not obvious, you say? Well, let's try to Saff Squeeze again, inlining closest_available
:
it "flies to the only available runway" do
Autopilot.stub(:current_position) { Position.new 0, 0 }
nearby_runway = create :runway, latitude: 1, longitude: 1
create :runway, city: nearby_runway.city, latitude: 2, longitude: 2
runways_in_city = runways_in city
runway = runways_in_city.select { |r| r.available? }
.sort_by { |r| Autopilot.distance_between Autopilot.current_position, r.position }.last
runway.should == nearby_runway
end
Now, where am I going to place a failing assertion higher in the test? I can't -- the bug is in the very last line of the test. Eventually I'll be forced to realize that it was in closest_available
before I inlined it.