ParallelTesting is vulnerable to race conditions

[melekhove]

Cleanup procedures performed in parallel results in exceptions on Windows at least.

Steps to reproduce:

sbt:dotty> testOnly dotty.tools.dotc.CompilationTests -- *testPlugins
[info] Test dotty.tools.dotc.CompilationTests.testPlugins started
[=======================================] completed (2/2, 0 failed, 13s)
[error] Test dotty.tools.dotc.CompilationTests.testPlugins failed: java.nio.file.FileSystemException: out\testPlugins\ne
g\divideZero-research\plugin.properties: The process cannot access the file because it is being used by another process.

[error] , took 13.749 sec
[error]     at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
[error]     at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
[error]     at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
[error]     at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
[error]     at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
[error]     at java.nio.file.Files.delete(Files.java:1126)
[error]     at dotty.tools.vulpix.ParallelTesting$CompilationTest.delete(ParallelTesting.scala:1109)
[error]     at dotty.tools.vulpix.ParallelTesting$CompilationTest.$anonfun$delete$2(ParallelTesting.scala:1108)
[error]     at dotty.tools.vulpix.ParallelTesting$CompilationTest.$anonfun$delete$2$adapted(ParallelTesting.scala:1108)
[error]     at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32)
[error]     at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29)
[error]     at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:194)
[error]     at dotty.tools.vulpix.ParallelTesting$CompilationTest.delete(ParallelTesting.scala:1108)
[error]     at dotty.tools.vulpix.ParallelTesting$CompilationTest.$anonfun$delete$1(ParallelTesting.scala:1105)
[error]     at dotty.tools.vulpix.ParallelTesting$CompilationTest.$anonfun$delete$1$adapted(ParallelTesting.scala:1105)
[error]     at scala.collection.immutable.List.foreach(List.scala:389)
[error]     at dotty.tools.vulpix.ParallelTesting$CompilationTest.delete(ParallelTesting.scala:1105)
[error]     at dotty.tools.vulpix.ParallelTesting$CompilationTest.cleanup(ParallelTesting.scala:1019)

================================================================================
Test Report
================================================================================

1 suites passed, 0 failed, 1 total
[error]     at dotty.tools.vulpix.ParallelTesting$CompilationTest.checkExpectedErrors(ParallelTesting.scala:985)

[error]     at dotty.tools.dotc.CompilationTests.testPlugins(CompilationTests.scala:295)
[error]     ...
[info] Test run finished: 1 failed, 0 ignored, 1 total, 14.341s
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error]         dotty.tools.dotc.CompilationTests
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] No tests to run for dotty-doc / Test / testOnly
[error] (dotty-compiler / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 17 s, completed 25.09.2018 8:49:05
sbt:dotty>

The correct implementation is not that easy I'm afraid.
Adding just new catch clause to

private def delete(file: JFile): Unit = {
  if (file.isDirectory) file.listFiles.foreach(delete)
  try Files.delete(file.toPath)
  catch {
    case _: NoSuchFileException => // already deleted, everything's fine
  }
}

will mute the issue, but I'm not sure it's the right way.

[melekhove]

I made an incorrect diagnostics and will create new issue.