Introduce extension API for container templates

[nipafx]

Test templates are a great way to create extensions that generate test cases per method. If an extension wants to generate suites of tests per class, though, this is not easily possible. I propose to implement a container template extension point that functions similarly to test template:

• it defines the number of executions n - but in this case per container, which makes Jupiter work all its "execute tests in this class" magic not once but n times
• each invocation is called with a context, which most importantly contains the index of the current iteration
• additional extensions can be registered

It might make sense to consider whether other extensions should be allowed to easily access the container invocation context, maybe from the ExtensionContext they already receive. (Thinking about that, wouldn't it make sense to do the same for test templates, i.e. give ExtensionContext a method that returns the invocation context?)

Where parameterized test methods were the canonical example of how to use test templates, theories would be the canonical example for container templates. With this extension point the theories extension would use reflection to find out how many parameters were defined and then have the class be executed so many times, each time dropping a different set of parameters into all test methods:

• a = 0; b = 0
  • testAddition(a, b)
  • testSubtraction(a, b)
• a = 0; b = 1
  • testAddition(a, b)
  • testSubtraction(a, b)
• ...

Related Issues

• Introduce support for parameterized classes #878
• Influence invocation order for TestTemplate executions #1141
• JUnit 4 parameterized support as an extension (#715) #723
• No runner equivalent #1239
• Generate multiple test instances (with names) from a TestInstanceFactory #2092

[marcphilipp]

Defining @ArgumentsSource on the test class (as proposed in #878) would be an equivalent solution, right?

[nipafx]

In parts. But it would still make the method the container for multiple tests instead of the class. And it would also require all methods to have the same parameters.