sbt build: the `junit` project is not compiled with the compiler from source code

[sjrd]

It apparently simply uses the stock Scala compiler fetched from Maven. The ant build uses the compiler compiled from source to compile the JUnit tests (in ant test.junit).

This means that, when fixing an issue in the compiler, and using a JUnit test to test the fix, sbt test fails while ant test.junit succeeds.

[retronym]

AFAICR, this was by design. Or rather, we realised that it would be hard to implement with our set of design principles for the SBT build (namely: bootstrapping was not something we wanted to handle internally in the build), and then convinced ourselves that this constraint was reasonable.

For tests of the new compiler, we either use partest, or, increasingly so, snippets of source code in strings in JUnit tests. The test case code itself should not rely on the latest compiler features.

[retronym]

The previous incarnation of the SBT build did handle bootstrapping internally, but we found this led to non-functional problems in SBT (performance, memory leaks).

[sjrd]

I understand why we do this to compile the standard library and the compiler. But it really seems weird that we don't use the freshly built compiler to compile the JUnit tests.

But well, if that's how it is, I yield ^^

[retronym]

closing, but /cc @adriaanm to confirm my recollection

[lrytz]

I agree this is an annoying problem, it keeps coming back often. your latest PR (scala/scala#5117) is a good example: its test will fail for people using sbt until we push a new starr (right?). the same happens for people running the JUnit tests within IntelliJ.

[sjrd]

Yes. That's why I used a partest instead of a JUnit test. test/it:testOnly -- ./test/files/run/t9516.scala passes from sbt, but test did not with a JUnit test.

[lrytz]

oh, i overlooked that you did a partest test.

[adriaanm]

On the bright side, this means we could more aggressively cache the compiled junit tests on the CI at some point :-)

It would still be good to have another project that uses junit to test the actual compiler, using the quick compiler to build the test sources. Not sure it's possible to switch ScalaInstances on a per-project basis, or maybe we should split out sbt build in two? One for building the compiler and one for building the tests using that compiler? The downside is that we diverge from common practice again then...