runtime: frequent timeouts in doTestParallelReaders since 2021-11-04

[bcmills]

greplogs --dashboard -md -l -e '(?ms)\ngoroutine \d+ \[run.*\]:\n(?:.+\n\t.*\n)*runtime_test\.doTestParallelReaders'

2021-11-18T06:05:29-f6647f2/linux-ppc64le-buildlet
2021-11-18T02:17:23-ce7e501/linux-ppc64-buildlet
2021-11-17T21:26:25-0440fb8/linux-ppc64le-power9osu
2021-11-17T17:36:36-0555ea3/linux-ppc64le-power9osu
2021-11-17T04:31:40-489f587/linux-ppc64le-power9osu
2021-11-17T04:31:22-f384c70/linux-ppc64le-power9osu
2021-11-16T18:36:59-6c36c33/linux-ppc64-buildlet
2021-11-16T07:47:08-313cae3/linux-ppc64le-power9osu
2021-11-15T21:22:14-42fa03a/linux-ppc64le-power9osu
2021-11-15T21:22:09-184ca3c/linux-ppc64le-power9osu
2021-11-13T00:26:24-787708a/linux-ppc64le-power9osu
2021-11-12T22:20:50-9150c16/linux-ppc64le-power9osu
2021-11-11T13:58:28-d76b1ac/linux-ppc64-buildlet
2021-11-11T05:16:39-99fa49e/linux-ppc64le-buildlet
2021-11-09T20:47:54-a096316/linux-ppc64le-power9osu
2021-11-09T19:01:06-81f37a7/linux-ppc64le-buildlet
2021-11-09T19:01:06-81f37a7/linux-ppc64le-power9osu
2021-11-08T21:52:47-955f9f5/linux-ppc64le-power9osu
2021-11-08T21:28:37-ccea0b2/linux-ppc64le-power9osu
2021-11-06T19:41:12-7ca772a/linux-ppc64le-buildlet
2021-11-05T20:59:43-dbd3cf8/linux-ppc64le-buildlet
2021-11-05T18:20:07-53bab19/linux-ppc64le-power9osu
2021-11-05T17:46:27-4f543b5/linux-ppc64-buildlet
2021-11-05T15:52:40-a0d661a/android-amd64-emu
2021-11-04T23:56:29-0e5f287/linux-ppc64le-buildlet
2021-11-04T21:50:21-bfd74fd/linux-ppc64le-buildlet
2021-11-04T20:01:22-6d1fffa/android-amd64-emu
2021-11-04T20:00:54-961aab2/illumos-amd64
2020-01-31T23:17:00-8390c47/android-amd64-emu
2019-11-01T05:38:51-e96fd13/netbsd-amd64-8_0

[bcmills]

Marking as release-blocker for Go 1.18 because the failure rate is markedly higher as of 2021-11-04: this looks like a regression, or at the very least a flaky test exacerbated by 1.18 changes.

(@mknyszek, another one for the “GC running during test” pile, perhaps?)

[mknyszek]

Oh, I think I see what's wrong here. The test already says that it's unsafe for a GC to run, and calls SetGCPercent(-1). However, I think what's missing here is that one could actively be in progress, so runtime.GC needs to be called to make sure.

I'll try to confirm it.

[gopherbot]

Change https://golang.org/cl/366534 mentions this issue: runtime: ensure no GC is running in TestParallelRWMutexReaders

[mknyszek]

I've had trouble reproducing the issue, but I do think my previous theory does indicate a real potential problem. That theory does also fit into the change in minimum heap size triggering the issue.

Sent golang.org/cl/366534. We can try landing it and see if it fixes it, placing this issue in a WaitingForInfo state. I think it's something that the test should be doing anyway.

[mknyszek]

Actually, I'm more confident this is actually the fix. If I simply start a GC on another goroutine before the test starts, it easily reproduces. I guess it's possible this is some other bug, but it's unlikely at this point. Once I land https://golang.org/cl/366534 I'll close this optimistically.

[gopherbot]

Change https://golang.org/cl/367874 mentions this issue: misc/cgo/test: further reduce likeliness of hang in Test9400