Allow @RepeatedTest to stop upon failure

[jaikiran]

Many a times when a particular test is a failing once in a while (i.e. not consistently), I use the @org.junit.jupiter.api.RepeatedTest with a relatively high number like 10:

@RepeatedTest(10)

and trigger the tests to try and reproduce the issue. Some of these 10 runs will fail (as expected). However, the only reason I used the @RepeatedTest was to reproduce a failure and once that failure has occurred, I'm not really interested in running the remaining number of "count" that was passed to it.

Would it be a good idea to introduce something like stopOnFailure attribute (which can be set to true, but defaults to false) to this @RepeatedTest so that it considers the value count as the maximum number of times to run the test and if stopOnFailure is set to true then stops as soon as the first run within those 10 runs fails?

[juliette-derancourt]

Tentatively slated for 5.6 M2 solely for the purpose of team discussion

[marcphilipp]

Team decision: Wait for additional interest from the community.

[antimpatel]

@marcphilipp can I pick this up?

[marcphilipp]

@antimpatel Please note that we haven't taken a decision whether we want to add and maintain this functionality. However, it would certainly help to see it spelled out in a spike. I think one of the main complications will be to change the execution mode to same_thread if the feature is used. You'll probably have to add a getExecutionMode() method or sth. similar to TestTemplateInvocationContextProvider.

I like this idea. The option to stop on failure could help test runs terminate quicker if there are faults.