How do you run SpecFlow scenarios from the command line using MSTest?

Now that SpecFlow 3.0 has been released we can use SpecFlow with .NET Core. The CLI tool for .NET Core is dotnet and tests are run like this if you use MSTest (vstest):

dotnet test

If the tests are in a specific project you can specify the project like this

dotnet test TestProject

where TestProject is the name of the project. You can skip the project name if you want to, but specifying it will make dotnet look in only that project. To list all the tests in the project you can use the -t flag:

dotnet test TestProject -t

To run only specific tests you can use the --filter flag:

dotnet test TestProject --filter ShouldBeSuccess_1

where ShouldBeSuccess_1 is the name of the test. The argument after --filter is an expression, and not necessary the name of the test If you had a test called ShouldBeSuccess_12 it would also run. You can see the rules for --filter here.

To only run the tests in a specific category you can use TestCategory:

dotnet test TestProject --filter TestCategory=ci

where ci is the category name. To add a test to a category you use tags.

To create the results file you have to use the --logger flag:

dotnet test TestProject --logger trx

Here it's used to create a trx results file.

Behind the scene specflow tests are just regular mstest unit tests. So you should be able to run them the same way using something like:

To run a specific scenario:

mstest /testcontainer:tests.dll /test:GivenMyScenarioWhenIDoSomeStuff

To run a several specific scenario you can use the /test flag multiple times:

mstest /testcontainer:tests.dll /test:GivenMyScenarioWhenIDoSomeStuff /test:GivenMyScenarioWhenIDoSomemthingElse

To run a feature

mstest /testcontainer:tests.dll /test:MyFeatureName

If you add tags on your scenarios using @MyTag for example, you could also use the option

/category:MyTag to filter down the scenarios to run.

Please have a look to the generated code behind of your feature files to get a clue of how things actually work, if you are familliar with mstest it should be pretty straightforward.