Common API for MockMvc tests and for real HTTP tests

[spring-projects-issues]

Neil S opened SPR-15081 and commented

There seems to be no common API which can be configure to do mock HTTP tests as well as real HTTP tests.

MockMvc tests suffice during development but for production deployment projects might like to do a complete end-to-end integration test by run real tests against a the (maybe embedded) container sever which will be used in production.

At the moment that means writing the same tests twice using the MockMvc and then TestRestTemplate+assert APIs.

IMHO MockMvc has the friendlier API which results in concise and easier to read test code.

---

Affects: 4.3.5

Issue Links:

• Extract MockHttp from MockMvc [SPR-13818] #18391 Extract MockHttp from MockMvc
• Support varargs for expectations in MockMvc [SPR-16417] #20963 Support varargs for expectations in MockMvc

0 votes, 6 watchers

[spring-projects-issues]

Rossen Stoyanchev commented

I see where you're coming from but for starters a name such as MockMvc would be a misnomer. More importantly by design it is a server-side test framework that simulates running in a Servlet container with the Spring MVC DispatcherServlet and that makes it possible to provide unique value such as whitebox testing -- e.g. assert things like model attributes, handled exceptions, the handler used, and others that couldn't be verified otherwise, and it also allows creating tests that are closer to unit testing such as testing a single controller at a time.

For an integration test with actual HTTP requests the client and server are much more decoupled and there is little unique value I can see us providing over existing tools such as REST Assured.

The general idea is that MockMvc allows you to test thoroughly your Spring MVC controllers and web framework configuration without running a server. Then you should need a lot less full integration testing.

[spring-projects-issues]

Neil S commented

Thanks, all good points.

I do think we ought to have some API which can test both server side and full end to end. Maybe that shouldn't be MockMvc... although I prefer its API over assert and feel that pushing everyone to TestRestTemplate would be a somewhat backward step.

How about a (generically named, say 'MvcTest'?) interface that MockMvc implements, which also has an implementation that makes real HTTP requests?

You're right that this wouldn't provide functionality not already available elsewhere but it would give the ability to reuse potentially huge amounts of code. At the moment I'm having to decide which tests are important enough to run e2e and duplicate the code using TestRestTemplate.

[spring-projects-issues]

Rossen Stoyanchev commented

This looks very promising. I did a quick experiment. Essentially just one extra class that wraps a RestTemplate and applies adaptation to and from the MockHttpServletRequest and MockHttpSevletResponse. Also an interface so test fixtures can be run through MockMvc or the RestTemplate-backed alternative.

I'm going to schedule tentatively for 5.0 RC1 where we are also planning to put together test support for the spring-web-reactive module and this idea here would be useful to flesh out at this time. Thanks for the suggestion!

[spring-projects-issues]

Rossen Stoyanchev commented

/cc robwinch and Andy Wilkinson

[spring-projects-issues]

Neil S commented

Great! Thanks very much :-)

[spring-projects-issues]

Rossen Stoyanchev commented

This needs to be considered together together with #18391 which comes from a different angle but is also about making MockMvc more broadly usable.

A separate concern that needs to be addressed is that many aspects of request building (request + session attributes + anything server-side specific) do not apply to integration tests. How do we deal with that is the question. We could fail fast or ignore those. The same concern is true for expectations matching (model attributes, handler method, etc) but there the matching would naturally fail..

[spring-projects-issues]

Rossen Stoyanchev commented

Note also that in 5.0 we have the WebTestClient which as actual client offers a MockMvc-like API that can be used with or without a server. Another route for addressing this ticket would be adapt it to work with both Spring MVC and WebFlux backends.

[inanemed]

Could you please confirm this is still an issue?

[molexx]

Yes.

@rstoyanchev 's experiment in #19647 (comment) looks to be a good solution.

[rstoyanchev]

I'll give this a try for 5.3 RC1.

[rstoyanchev]

After some more thought, while my earlier experiment in #19647 (comment) does work, it has a major issue in that MockMvc is for server tests and exposes lots of request inputs and response assertions that cannot be passed over HTTP such as request and session attributes, contextPath, servletPath, model attributes, and many others. It makes it a poor choice for a common API for both mock HTTP and real HTTP tests.

A better path is via WebTestClient which already supports a common API for mock HTTP tests (with WebFlux) and for real HTTP tests. We can add support for mock HTTP tests (with Spring MVC) targeting MockMvc as the server. This support will expose equivalent setup options as are available in MockMvc, including standalone and ApplicationContext setups, adding filters, setup extension hooks (e.g. Spring Security setup), etc. but actual tests will be performed through the WebTestClient.

Among the benefits of this approach, as an actual HTTP client the WebTestClient provides encoding and decoding of request and response content (i.e. message conversion) which is convenient for assertions against higher level objects. For all other assertions in MockMvc, the WebTestClient offers equivalents but through a fluent API without static imports. Async requests will be a little easier to do through WebTestClient because the asyncDispatch is performed transparently.

I've experimented locally and it is fairly straight forward to do. I should have something concrete this week.

What does it mean for existing MockMvc tests and MockMvc? WebTestClient will use MockMvc to perform requests so MockMvc remains as the foundation for Spring MVC testing. I do expect however that a majority of tests will now be performed through the WebTestClient. Existing tests can remain as is, or it's easy to re-write them as well to WebTestClient.

[molexx]

Sounds fair to me, thanks.

I've not looked at WebTestClient so far because we don't use Webflux, looking forward to checking it out.

[rstoyanchev]

The new MockMvcTestClient is now available in master with 5.3 snapshots. I've ported all existing sample tests which can be used for comparison.

Before:

• org/springframework/test/web/servlet/samples/standalone
• org/springframework/test/web/servlet/samples/context

After

• org/springframework/test/web/servlet/samples/client/standalone
• org/springframework/test/web/servlet/samples/client/context

I am keeping the issue open until I've the documentation has been updated.

I've not looked at WebTestClient so far because we don't use Webflux

Use of WebTestClient requires hardly any knowledge of WebFlux. You'll see from the before and after tests the experience is largely the same. I've even managed to keep the ability to perform MockMvc assertions on the server response for those extra things like model attributes, flash attributes, etc.

[molexx]

This looks great, thanks!

I think it would be useful to add an example that shows best practise to also run the tests from WebAppResourceTests in http mode please.

I'm not sure what the neatest way is? Thinking aloud, perhaps the tests could be in an abstract class that is extended by two concrete test classes - one setting up the MockMVCTestClient as shown and the other starting the http server with @SpringBootTest(webEnvironment=...) and leaving the WebTestClient to be autoconfigured. Or maybe the same test class could configure itself differently depending on a property passed at test runtime?

[rstoyanchev]

I think it would be useful to add an example that shows best practise to also run the tests from WebAppResourceTests in http mode please.

Replace the test client initialization with something like this (and you'll also need to run the server):

this.testClient = WebTestClient.bindToServer().baseUrl("http://localhost:8080").build();

perhaps the tests could be in an abstract class that is extended by two concrete test classes - one setting up the MockMVCTestClient as shown and the other starting the http server with @SpringBootTest(webEnvironment=...) and leaving the WebTestClient to be autoconfigured.

Yes that's one way to do it. I suppose it depends on whether you want to run both every time or selectively. It probably makes sense to run in MockMvc mode most often and in live server mode once in a while. If setting things up programmatically you could use JUnit's Assume to check some environment variable or @ParameterizedTest to run both. Or in a more declarative style with Boot's @SpringBootTest(webEnvironment=...), I think JUnit's @EnabledIfEnvironmentVariable would be handy.

[molexx]

Great thanks.

I think it's worth mentioning this in the docs to make it clear how easy it is to do.

[nvora]

Do we have a reference project with an example of testing rest api with both and live server? if yes please share