Run single test against multiple configurations in Visual Studio
Refactor the test startup to allow for it to be modified as needed for its test
For example
public class TestStartup : IStartup {
private readonly string settings;
public TestStartup(string settings) {
this.settings = settings;
}
public void ConfigureServices(IServiceCollection services) {
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile(settings, false) //<--just an example
.AddEnvironmentVariables()
.Build();
services.AddMvc()
.SetCompatibilityVersion(version: CompatibilityVersion.Version_2_2);
//...Code to add required services based on configuration
}
public void Configure(IApplicationBuilder app) {
app.UseMvc();
//...Code to configure test Startup
}
}
And have that pattern filter up through the fixture
public class TestServerFixture {
static readonly Dictionary<string, TestServer> cache =
new Dictionary<string, TestServer>();
public TestServerFixture() {
//...
}
public HttpClient GetClient(string settings) {
TestServer server = null;
if(!cache.TryGetValue(settings, out server)) {
var startup = new TestStartup(settings); //<---
var builder = new WebHostBuilder()
.ConfigureServices(services => {
services.AddSingleton<IStartup>(startup);
});
server = new TestServer(builder);
cache.Add(settings, server);
}
return server.CreateClient();
}
}
And eventually the test itself
public class MyTest : IClassFixture<TestServerFixture> {
private readonly TestServerFixture fixture;
public MyTest(TestServerFixture fixture) {
this.fixture = fixture;
}
[Theory]
[InlineData("settings1.json")]
[InlineData("settings2.json")]
public async Task Should_Execute_Using_Configurations(string settings) {
var client = fixture.CreateClient(settings);
//...use client
}
}
@Nkosi's post fits very well with our scenario and my asked question. It's a simple, clean and easy to understand approach with maximum reusability. Full marks to the answer.
However, there were a few reasons why I could not go forward with the approach:
In the suggested approach we couldn't run tests for only one particular
setting
. The reason it was important for us as in the future, there could two different teams maintaining their specific implementation and deployment. WithTheory
, it becomes slightly difficult to run only onesetting
for all the tests.There is a high probability that we may need two separate build and deployment pipelines for each setting/ deployment.
While the API endpoints,
Request
, andResponse
are absolutely the same today, we do not know if it will continue to be the case as our development proceed.
Due to the above reasons we also considered the following two approaches:
Approach 1
Have a common class
library which has common Fixture
and Tests
as abstract
class
- Project Common.IntegrationTests
TestStartup.cs
public abstract class TestStartup : IStartup
{
public abstract IServiceProvider ConfigureServices(IServiceCollection services);
public void Configure(IApplicationBuilder app)
{
app.UseMvc();
// Code to configure test Startup
}
}
TestServerFixture.cs
public abstract class TestServerFixture
{
protected TestServerFixture(IStartup startup)
{
var builder = new WebHostBuilder().ConfigureServices(services =>
{
services.AddSingleton<IStartup>(startup);
});
var server = new TestServer(builder);
Client = server.CreateClient();
}
public HttpClient Client { get; private set; }
}
MyTest.cs
public abstract class MyTest
{
private readonly TestServerFixture _fixture;
protected MyTest(TestServerFixture fixture)
{
_fixture = fixture;
}
[Fact]
public void ItShouldExecuteTwice_AgainstTwoSeparateConfigurations()
{
//...
}
}
- Project Setting1.IntegrationTests (References Common.IntegrationTests)
TestStartup.cs
public class TestStartup : Common.IntegrationTests.TestStartup
{
public override IServiceProvider ConfigureServices(IServiceCollection services)
{
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", false) // appsettings for Setting1
.AddEnvironmentVariables()
.Build();
services.AddMvc()
.SetCompatibilityVersion(version: CompatibilityVersion.Version_2_2);
// Code to add required services based on configuration
return services.BuildServiceProvider();
}
}
TestServerFixture.cs
public class TestServerFixture : Fixtures.TestServerFixture
{
public TestServerFixture() : base(new TestStartup())
{
}
}
MyTests.cs
public class MyTests : Common.IntegrationTests.MyTests, IClassFixture<TestServerFixture>
{
public MyTests(TestServerFixture fixture) : base(fixture)
{
}
}
- Project Setting2.IntegrationTests (References Common.IntegrationTests)
A similar structure as Setting1.IntegrationTests
This approach provided a good balance of reusability and flexibility to run/ modify the tests independently. However, I was still not 100% convinced with this approach as it meant for each common Test
class we would need to have an implementation where we are not doing anything other than calling the base
constructor
.
Approach 2
In the second approach, we took the Approach 1 further and try to fix the issue we had with Approach 1 with Shared Project. From the documentation:
Shared Projects let you write common code that is referenced by a number of different application projects. The code is compiled as part of each referencing project and can include compiler directives to help incorporate platform-specific functionality into the shared code base.
Shared Project gave us the best of both worlds without the ugliness of link
files and unnecessary class inheritance
or abstraction
. Our new set up is as follows:
Edit: I wrote a blog post on this where I have talked about our use-case and the solution in detail. Here is the link:
https://ankitvijay.net/2020/01/04/running-an-asp-net-core-application-against-multiple-db-providers-part-2/