Controlling execution order of unit tests in Visual Studio

As you should know by now, purists say it's forbiden to run ordered tests. That might be true for unit tests. MSTest and other Unit Test frameworks are used to run pure unit test but also UI tests, full integration tests, you name it. Maybe we shouldn't call them Unit Test frameworks, or maybe we should and use them according to our needs. That's what most people do anyway.

I'm running VS2015 and I MUST run tests in a given order because I'm running UI tests (Selenium).

Priority - Doesn't do anything at all This attribute is not used by the test system. It is provided to the user for custom purposes.

orderedtest - it works but I don't recommend it because:

1. An orderedtest a text file that lists your tests in the order they should be executed. If you change a method name, you must fix the file.
2. The test execution order is respected inside a class. You can't order which class executes its tests first.
3. An orderedtest file is bound to a configuration, either Debug or Release
4. You can have several orderedtest files but a given method can not be repeated in different orderedtest files. So you can't have one orderedtest file for Debug and another for Release.

Other suggestions in this thread are interesting but you loose the ability to follow the test progress on Test Explorer.

You are left with the solution that purist will advise against, but in fact is the solution that works: sort by declaration order.

The MSTest executor uses an interop that manages to get the declaration order and this trick will work until Microsoft changes the test executor code.

This means the test method that is declared in the first place executes before the one that is declared in second place, etc.

To make your life easier, the declaration order should match the alphabetical order that is is shown in the Test Explorer.

• A010_FirstTest
• A020_SecondTest
• etc
• A100_TenthTest

I strongly suggest some old and tested rules:

• use a step of 10 because you will need to insert a test method later on
• avoid the need to renumber your tests by using a generous step between test numbers
• use 3 digits to number your tests if you are running more than 10 tests
• use 4 digits to number your tests if you are running more than 100 tests

VERY IMPORTANT

In order to execute the tests by the declaration order, you must use Run All in the Test Explorer.

Say you have 3 test classes (in my case tests for Chrome, Firefox and Edge). If you select a given class and right click Run Selected Tests it usually starts by executing the method declared in the last place.

Again, as I said before, declared order and listed order should match or else you'll in big trouble in no time.

You can Use Playlist

Right click on the test method -> Add to playlist -> New playlist

the execution order will be as you add them to the playlist but if you want to change it you have the file

Merge your tests into one giant test will work. To make the test method more readable, you can do something like

[TestMethod]
public void MyIntegratonTestLikeUnitTest()
{
    AssertScenarioA();

    AssertScenarioB();

    ....
}

private void AssertScenarioA()
{
     // Assert
}

private void AssertScenarioB()
{
     // Assert
}

Actually the issue you have suggests you probably should improve the testability of the implementation.

I dont see anyone mentioning the ClassInitialize attribute method. The attributes are pretty straight forward.

Create methods that are marked with either the [ClassInitialize()] or [TestInitialize()] attribute to prepare aspects of the environment in which your unit test will run. The purpose of this is to establish a known state for running your unit test. For example, you may use the [ClassInitialize()] or the [TestInitialize()] method to copy, alter, or create certain data files that your test will use.

Create methods that are marked with either the [ClassCleanup()] or [TestCleanUp{}] attribute to return the environment to a known state after a test has run. This might mean the deletion of files in folders or the return of a database to a known state. An example of this is to reset an inventory database to an initial state after testing a method that is used in an order-entry application.

• [ClassInitialize()] Use ClassInitialize to run code before you run the first test in the class.

• [ClassCleanUp()] Use ClassCleanup to run code after all tests in a class have run.

• [TestInitialize()] Use TestInitialize to run code before you run each test.

• [TestCleanUp()] Use TestCleanup to run code after each test has run.