x/tools/internal/lsp/regtest: various flakes due to go.mod contention

[bcmills]

https://build.golang.org/log/fe27a791913be11bc439f9612e1ba92f52673e54

2020/02/20 00:34:56 initial workspace load failed: go [list -f {{context.GOARCH}} {{context.Compiler}} -modfile=/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/tmp/go.042197626.mod -- unsafe]: exit status 1: go: open /var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/tmp/go.042197626.mod: no such file or directory

2020/02/20 00:35:01 diagnose: could not generate diagnostics for go.mod file: err: chdir /var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/tmp/goplstest-ws-lsprpc-132098964: no such file or directory: stderr: 
2020/02/20 00:35:04 diagnose: could not generate diagnostics for go.mod file: err: exit status 1: stderr: go: cannot find main module, but -modfile was set.
	-modfile cannot be used to set the module root directory.

2020/02/20 00:35:07 diagnose: could not generate diagnostics for go.mod file: err: exit status 1: stderr: go: open /var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/tmp/go.115179283.mod: no such file or directory

2020/02/20 00:35:08 serving stream: failed reading header line "EOF"
2020/02/20 00:35:37 : context deadline exceeded
--- FAIL: TestGoToStdlibDefinition (37.17s)
    --- FAIL: TestGoToStdlibDefinition/forwarded (30.41s)
        definition_test.go:69: Definition: context deadline exceeded
panic: Shutdown: context deadline exceeded [recovered]
	panic: Shutdown: context deadline exceeded

goroutine 11620 [running]:
testing.tRunner.func1.1(0x1698620, 0xc00c71a0b0)
	/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/testing/testing.go:941 +0x3d0
testing.tRunner.func1(0xc00a536360)
	/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/testing/testing.go:944 +0x3f9
panic(0x1698620, 0xc00c71a0b0)
	/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/runtime/panic.go:967 +0x15d
golang.org/x/tools/internal/lsp/regtest.(*Runner).RunInMode.func1.1(0xc00a1fb140, 0x189f140, 0xc00a1fad80)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/gopath/src/golang.org/x/tools/internal/lsp/regtest/env.go:178 +0x70
runtime.Goexit()
	/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/runtime/panic.go:615 +0x192
testing.(*common).FailNow(0xc00a536360)
	/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/testing/testing.go:657 +0x39
testing.(*common).Fatal(0xc00a536360, 0xc000a3bd98, 0x1, 0x1)
	/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/testing/testing.go:718 +0x78
golang.org/x/tools/internal/lsp/regtest.(*Env).GoToDefinition(0xc00a1fb140, 0x17774e5, 0x7, 0x8, 0x13, 0x100ea48, 0x10, 0x16c56a0, 0x173f301)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/gopath/src/golang.org/x/tools/internal/lsp/regtest/env.go:310 +0x130
golang.org/x/tools/internal/lsp/regtest.TestGoToStdlibDefinition.func1(0x189f140, 0xc00a1fad80, 0xc00a536360, 0xc00a1fb140)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/gopath/src/golang.org/x/tools/internal/lsp/regtest/definition_test.go:69 +0x8d
golang.org/x/tools/internal/lsp/regtest.(*Runner).RunInMode.func1(0xc00a536360)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/gopath/src/golang.org/x/tools/internal/lsp/regtest/env.go:181 +0x2af
testing.tRunner(0xc00a536360, 0xc00a508500)
	/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/testing/testing.go:992 +0xdc
created by testing.(*T).Run
	/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/testing/testing.go:1043 +0x357
FAIL	golang.org/x/tools/internal/lsp/regtest	44.585s

CC @stamblerre @jayconrod @matloob

[stamblerre]

@ridersofrohan: Can you take a look at these failures? The error messages at the top seem to be caused by -modfile.

[findleyr]

Thanks, this looks like a flake. Let me investigate first.

[findleyr]

Ok, I think this is a test simply timing out due to an extremely slow initial workspace load (we delete the temp modfiles when the test exits, so I think the -modfile error log is a red herring).

Locally, the entire regtest suite runs in ~3s, but I can produce a similar error message by setting the test timeout to 1s.

As a temporary fix, I'm going to increase the timeout to 60s to see if this still flakes. Longer term, I need to understand why this test runs more than an order of magnitude slower on the builders.

[gopherbot]

Change https://golang.org/cl/220280 mentions this issue: internal/lsp/regtest: increase test timeout to 60s

[bcmills]

Here is a similar failure on freebsd-amd64-race, in TestGoToStdlibDefinition/singleton in
https://build.golang.org/log/a7e57b48b2eb87387651b04e268638d5b1b25f9d

--- FAIL: TestGoToStdlibDefinition (60.55s)
    --- FAIL: TestGoToStdlibDefinition/singleton (60.53s)
        definition_test.go:69: Definition: context deadline exceeded
panic: Shutdown: context deadline exceeded [recovered]
	panic: Shutdown: context deadline exceeded

goroutine 32 [running]:
testing.tRunner.func1.1(0xe31000, 0xc0024ade50)
	/tmp/workdir/go/src/testing/testing.go:941 +0x5d0
testing.tRunner.func1(0xc000224fc0)
	/tmp/workdir/go/src/testing/testing.go:944 +0x600
panic(0xe31000, 0xc0024ade50)
	/tmp/workdir/go/src/runtime/panic.go:973 +0x396
golang.org/x/tools/internal/lsp/regtest.(*Runner).RunInMode.func1.1(0xc00028e2a0, 0x1087780, 0xc000254120)
	/tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/regtest/env.go:178 +0xc1
runtime.Goexit()
	/tmp/workdir/go/src/runtime/panic.go:634 +0x104
testing.(*common).FailNow(0xc000224fc0)
	/tmp/workdir/go/src/testing/testing.go:657 +0x5b
testing.(*common).Fatal(0xc000224fc0, 0xc0001f9ba0, 0x1, 0x1)
	/tmp/workdir/go/src/testing/testing.go:718 +0x8a
golang.org/x/tools/internal/lsp/regtest.(*Env).GoToDefinition(0xc00028e2a0, 0xf16f3a, 0x7, 0x8, 0x13, 0xc0002283c0, 0xc000254300, 0xc000268000, 0x0)
	/tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/regtest/env.go:310 +0x1a2
golang.org/x/tools/internal/lsp/regtest.TestGoToStdlibDefinition.func1(0x1087780, 0xc000254120, 0xc000224fc0, 0xc00028e2a0)
	/tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/regtest/definition_test.go:69 +0x9e
golang.org/x/tools/internal/lsp/regtest.(*Runner).RunInMode.func1(0xc000224fc0)
	/tmp/workdir/gopath/src/golang.org/x/tools/internal/lsp/regtest/env.go:181 +0x3bc
testing.tRunner(0xc000224fc0, 0xc000228230)
	/tmp/workdir/go/src/testing/testing.go:992 +0x1ec
created by testing.(*T).Run
	/tmp/workdir/go/src/testing/testing.go:1043 +0x661
FAIL	golang.org/x/tools/internal/lsp/regtest	61.597s

[bcmills]

And that appears to be after the timeout increase: note that the time has risen to 60s.

I wonder if the underlying problem is a failure to propagate some other error appropriately. (See #36444 and #36605.)

[findleyr]

Hmm, my intuition was that the timeout was not specific to this test, but the fact that it is the same test and that it timed out at 60s suggests that this is a real failure. I haven't yet been able to reproduce, but I will skip this test until it is understood.

[gopherbot]

Change https://golang.org/cl/220360 mentions this issue: internal/lsp/regtest: skip flaky TestGoToStdlibDefinition