Document @DynamicPropertySource use cases that require @DirtiesContext

[philwebb]

It appears that @DynamicPropertySource cannot be used on a test superclass. See https://twitter.com/aheritier/status/1311721275811917824

[aheritier]

Effectively @philwebb

I am using Spring Boot 2.3.4.RELEASE with testcontainers 1.15.0-rc2 (but I don't think it has any impact here.)

My tests are using JUnit 5 and I was creating a parent/base class for all my tests suites using the container as described in: https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto-testcontainers

With such approach only the tests executed in the first test class are working and the next ones are failing.

@container and @DynamicPropertySource in a parent class

Only the the 1st test class is succeeding because the configuration isn't updated for others tests

2020-10-01 19:21:13.966 INFO 52376 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32848
19:21:02.888 [main] INFO org.testcontainers.containers.wait.strategy.HttpWaitStrategy - /mystifying_joliot: Waiting for 120 seconds for URL: http://localhost:32848/
2020-10-01 19:21:26.781  INFO 52376 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /boring_ganguly: Waiting for 120 seconds for URL: http://localhost:32850/
2020-10-01 19:21:37.942  INFO 52376 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /condescending_galileo: Waiting for 120 seconds for URL: http://localhost:32852/
2020-10-01 19:21:49.065  INFO 52376 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /upbeat_neumann: Waiting for 120 seconds for URL: http://localhost:32854/
2020-10-01 19:22:00.284  INFO 52376 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /compassionate_lamarr: Waiting for 120 seconds for URL: http://localhost:32856/
2020-10-01 19:22:11.489  INFO 52376 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /jolly_keller: Waiting for 120 seconds for URL: http://localhost:32858/

@container in a parent class and @DynamicPropertySource in every test class

It's ok

2020-10-01 19:27:08.172 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32861
19:26:58.269 [main] INFO org.testcontainers.containers.wait.strategy.HttpWaitStrategy - /elegant_wu: Waiting for 120 seconds for URL: http://localhost:32861/
2020-10-01 19:27:31.272 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32863
2020-10-01 19:27:22.046  INFO 55726 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /pensive_nightingale: Waiting for 120 seconds for URL: http://localhost:32863/
2020-10-01 19:27:49.474 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32865
2020-10-01 19:27:40.219  INFO 55726 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /epic_haslett: Waiting for 120 seconds for URL: http://localhost:32865/
2020-10-01 19:28:07.399 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32867
2020-10-01 19:27:57.138  INFO 55726 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /angry_gagarin: Waiting for 120 seconds for URL: http://localhost:32867/
2020-10-01 19:28:24.925 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32869
2020-10-01 19:28:15.596  INFO 55726 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /competent_jennings: Waiting for 120 seconds for URL: http://localhost:32869/
2020-10-01 19:28:43.334 INFO 55726 --- [           main] c.c.s.analytics.ElasticsearchTestCase    : spring.elasticsearch.rest.uris:localhost:32871
2020-10-01 19:28:34.020  INFO 55726 --- [           main] o.t.c.wait.strategy.HttpWaitStrategy     : /hardcore_burnell: Waiting for 120 seconds for URL: http://localhost:32871/

It seems that @DynamicPropertySource is called only once when several classes are extending a base class with this annotation.

HTH

[sbrannen]

What happens if you annotate your base test class with @DirtiesContext?

[sbrannen]

What happens if you annotate your base test class with @DirtiesContext?

Doing that will cause a new ApplicationContext to be loaded for each subclass, which is really what you want since the Spring environment properties are changing for each subclass (i.e., the port in the example).

---

I have a local fix in place that tracks the testClass in DynamicPropertiesContextCustomizer in addition to the methods. The actual fix is then an equality check added to equals() like the following.

public boolean equals(Object obj) {
	if (this == obj) {
		return true;
	}
	if (obj == null || getClass() != obj.getClass()) {
		return false;
	}
	DynamicPropertiesContextCustomizer that = (DynamicPropertiesContextCustomizer) obj;
	return this.testClass.equals(that.testClass) && this.methods.equals(that.methods);
}

This change forces a new ApplicationContext to be created for subclasses of a test class that uses a @DynamicPropertySource. I think this makes sense in general.

@philwebb, what are your thoughts on the matter?

1. track the testClass as above?
2. stop comparing the methods in equals() (and just compare the testClass)?
3. any other options?

[aheritier]

Hi @sbrannen

Thanks a lot for looking at this issue.

I tested and confirm that using @DirtiesContext in the parent class is solving the issue.

Not sure if the behaviour of @DynamicPropertySource should be changed/fixed maybe there are some legit use cases where it's not wanted/needed to reset the context.

Cheers

[sbrannen]

Thanks a lot for looking at this issue.

de rien (you're welcome)

I tested and confirm that using @DirtiesContext in the parent class is solving the issue.

Thanks for confirming.

Not sure if the behaviour of @DynamicPropertySource should be changed/fixed maybe there are some legit use cases where it's not wanted/needed to reset the context.

Yeah, I have also been pondering that. In the end it depends on whether the dynamic properties change for every lookup. In your case, they do: you get a different port number every time. But in other scenarios, the dynamic properties might not be that dynamic: they might rather just be unknown but unchanging for the current environment.

With that in mind, I'm considering converting this into a documentation issue.

@philwebb, thoughts?

[philwebb]

Is there a sample with some code I can look at? I'm not sure I've totally got a handle on the setup.