Integration tests are not working if using inherited Startup and reference to Microsoft.AspNetCore.App without version specified

[satano]

Describe the bug

We use integration tests in our AspNetCore application. We use inherited Startup class in our test project, so we can easily modify things in it for the testing purposes. The SUT is web API project that references Microsoft.AspNetCore.App. Everything works, if this reference has version specified. If we remove the version specification, the tests stop working. Requests to the test server keep returning 404 NotFound.

Version specification of this package is not recommended. If it is there, the compiler generates warning:

Warning NETSDK1071 A PackageReference to 'Microsoft.AspNetCore.App' specified a Version of 2.2.0. Specifying the version of this package is not recommended. For more information, see https://aka.ms/sdkimplicitrefs

Integration tests are always working (regardless of reference version specification) if they directly use Startup class from SUT.

To Reproduce

1. Create a simple web API project. Make sure, the reference to Microsoft.AspNetCore.App in .csproj is without version specified: <PackageReference Include="Microsoft.AspNetCore.App" />
2. Create integration tests project, that references web API project.
3. Create a Startup class in test project, which is inherited form API's project Startup.
4. Make a simple test which uses test Startup and makes a request to correct API in test server.
5. The request should return 200 OK, but returns 404 NotFound.
6. Modify the refeference in API project to have version specified: <PackageReference Include="Microsoft.AspNetCore.App" Version="2.2.0" /> The test will work.

Expected behavior

Integration tests should work as expected even if the package is without version.

Additional context

I created a full (yet simple) demo repository so you can clone it, run the tests and see the results.

There are two web API projects, which are the same. The only difference is how the Microsoft.AspNetCore.App package is referenced. And there are 4 sets of 2 test projects, where one project references API with version, the other test project API without version. 4 sets are because I used different approaches:

• Direct using of Startup class in API project. In this case, always everything works.
• Using inherited Startup class. Non of these approaches works with API project without version.
  • Using this inherited class directly.
  • Using inherited WebApplicationFactory.
  • Using TestServer.

[javiercn]

I believe there are some prerequisites missing. You can take a look at this link
https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-2.2#test-app-prerequisites

and see if that solves your issue.

[satano]

@javiercn No, this is not an issue. I have all the prerequisites. The projects with tests and the SUT projects are completely the same. The only difference is in the package reference in the SUT projects:

• <PackageReference Include="Microsoft.AspNetCore.App" Version="2.2.0" /> - the tests are working.
• <PackageReference Include="Microsoft.AspNetCore.App" /> - the tests are not working.

Package references in the projects with the tests are the same - without version specification. So if the tests work or not depends only on if there is a version specification of Microsoft.AspNetCore.App in tested project or there is not.

[javiercn]

Look at
https://github.com/satano/WebApiIntegrationTests/blob/master/TestServerTests/TestServerTests.csproj#L1

From the prerequisites section:
Specify the Web SDK in the project file (). The Web SDK is required when referencing the Microsoft.AspNetCore.App metapackage.

[satano]

OK. My fault - I am sorry. I looked only at the tested project (SUT) and there is correct SDK. But anyway, this still is not the issue. I changed it in all the projects and the results are the same.

But even if this would have solved it, it would be strange. Because really, the tests are working even if the SDK of test project is just Microsoft.NET.Sdk. They will stop working if I remove Version="x.y.z" from the package reference in the SUT. No other changes are made and this change is only in the SUT project - it does not matter if version is specified or not in the test project.

[javiercn]

I'm pretty sure it's an issue with dependency context. There's likely a mismatch between the version resolved in the app vs the version resolved in the test.

[satano]

Probably it is. We need inherited Startup class in test project and we do not want that warning about version. I will do anything what will help - just do not know what to do.

The strange thing (for me) is that the tests are working correctly even if the package version is not specified if you use Startup class directly from SUT.

[Eilon]

@natemcmaster can you try to repro this and see what's going on?

[natemcmaster]

I can confirm what @satano has reported - MVC is not discovering the controllers as expected. The source of the problem is one we've seen in a variety of ways before, and as @javiercn guessed, it is a problem with interpreting dependency context.

Context: when user code calls .AddMvc() inside Startup.ConfigureServices, MVC attempts to answer the question, "which assemblies contain controllers?" It does this through a variety of methods, one of which includes reading the .deps.json file to determine if the assembly references Microsoft.AspNetCore.Mvc. This eventually calls into this code: https://github.com/aspnet/AspNetCore/blob/release/2.2/src/Mvc/Mvc.Core/src/ApplicationParts/ApplicationAssembliesProvider.cs

What's broken: Because of the way the .NET Core SDK implemented versionless PackageReference, the .deps.json file is generated with the dependency information omitted. In @satano's repro, this is visible if you compare InheritedStartupTests/bin/Debug/netcoreapp2.2/InheritedStartupTests.deps.json and InheritedStartupTests_WithVersion/bin/Debug/netcoreapp2.2/InheritedStartupTests_WithVersion.deps.json. On the left, the .deps.json generated when using the SDK as directed. On the right, the result of adding the Version attribute.

In the scenario on the left (the versionless package ref), MVC skips controller discover on the app's entry assembly because it does not think MVC is used.

One potential workaround:
You could try manually telling MVC which assemblies have controllers by using ConfigureApplicationPartManager like this.

        public void ConfigureServices(IServiceCollection services)
        {
            services
                .AddMvc()
                .SetCompatibilityVersion(CompatibilityVersion.Version_2_2)
                .ConfigureApplicationPartManager(p =>
                {
                    var assembly = typeof(Startup).Assembly;
                    var partFactory = ApplicationPartFactory.GetApplicationPartFactory(assembly);
                    foreach (var part in partFactory.GetApplicationParts(assembly))
                    {
                        p.ApplicationParts.Add(part);
                    }
                });
        }

cc @javiercn @pranavkm

[satano]

@natemcmaster thank you for your investigation and workaround.

I have one more questions - basically just for me to understand it more. The .deps.json files are generated this way also for projects which references Startup class directly from SUT (ProjectStartupTests...). There is no "Microsoft.AspNetCore.App": "2.2.0" dependency in ProjectStartupTests.deps.json, but the tests are working. Why?