Is there a way to make Eclipse run a JUnit test multiple times until failure?

Here is a post I wrote that shows several ways of running the tests repeatedly with code examples: http://codehowtos.blogspot.com/2011/04/run-junit-test-repeatedly.html

You can use the @Parametrized runner, or use the special runner included in the post There is also a reference to a @Retry implementation

If you really want to run a test class until failure, you need your own runner.

@RunWith(RunUntilFailure.class)
public class YourClass {

    // ....

}

which could be implemented as follows...

package com.example;

import org.junit.internal.runners.*;
import org.junit.runner.notification.*;
import org.junit.runner.*;

public class RunUntilFailure extends Runner {

    private TestClassRunner runner;

    public RunUntilFailure(Class<?> klass) throws InitializationError {
        this.runner = new TestClassRunner(klass);
    }

    @Override
    public Description getDescription() {
        Description description = Description.createSuiteDescription("Run until failure");
        description.addChild(runner.getDescription());
        return description;
    }

    @Override
    public void run(RunNotifier notifier) {
        class L extends RunListener {
            boolean fail = false;
            public void testFailure(Failure failure) throws Exception { fail = true; }
        }
        L listener = new L();
        notifier.addListener(listener);
        while (!listener.fail) runner.run(notifier);
    }

}

...releasing untested code, feeling TDD guilt :)

If the for loop works, then I agree with nos.

If you need to repeat the entire setup-test-teardown, then you can use a TestSuite:

1. Right-click on the package containing the test to repeat
2. Go to New and choose to create a JUnit test SUITE
3. Make sure that only the test you want to repeat is selected and click through to finish.
4. Edit the file to run it multiple times.

In the file you just find the

addTestSuite(YourTestClass.class)

line, and wrap that in a for loop.

I'm pretty sure that you can use addTest instead of addTestSuite to get it to only run one test from that class if you just want to repeat a single test method.

I know it doesn't answer the question directly but if a test isn't passing every time it is run it is a test smell known as Erratic Test. There are several possible causes for this (from xUnit Test Patterns):

• Interacting Tests
• Interacting Test Suites
• Lonely Test
• Resource Leakage
• Resource Optimism
• Unrepeatable Test
• Test Run War
• Nondeterministic Test

The details of each of these is documented in Chapter 16 of xUnit Test Patterns.