x/tools: frequent out-of-memory errors on linux-arm builder

[bcmills]

The tests under golang.org/x/tools almost always fail on the linux-arm builder.

Looking at the build dashboard, the most frequent failure mode (by far) seems to result in errors of the form fork/exec /workdir/go/bin/go: cannot allocate memory *os.PathError in the golang.org/x/tools/internal/lsp tests:

https://build.golang.org/log/5fb7985054d8b682b81438c385ff16e8325a3aa6
https://build.golang.org/log/e422caa59d35f5c76e097fdadb60407ee2807368
https://build.golang.org/log/3e3e538cf33e70880da37c2dd1c716b733b4b009
https://build.golang.org/log/739f069936cc47cc800b4d898c505e17a9d4a7e1

However, some of the other tests have also flaked with various resource-exhaustion errors:

A similar cannot allocate memory failure mode for golang.org/x/tools/go/internal/gcimporter:
https://build.golang.org/log/99a21d9e2e1064d08890439b43de987e0bfcdf52

fatal error: runtime: out of memory in golang.org/x/tools/go/ssa:
https://build.golang.org/log/de56b0ffe0e39b38b9ed449bd0cd382be72eda69

pthread_create failed: Resource temporarily unavailable and no space left on device:
https://build.golang.org/log/7151dca16e38a6f61051f032b2f6b713253052ac

---

I don't know exactly what the resource footprint of the linux-arm builder is (@andybons or @dmitshur might know?), but if the x/tools tests are expected to pass within that footprint we may need to dial back the test concurrency on that builder.

CC @ianthehat

[dmitshur]

I don't know exactly what the resource footprint of the linux-arm builder is

Our linux/arm builders are on Scaleway, see here. There are 50 builders and they're C1 BareMetal instances. Documentation at https://www.scaleway.com/en/bare-metal-instances/ says C1 instances have 2 GB of RAM, 4 ARMv7 cores.

2 GB of RAM certainly isn't a lot. The next step up, C2S, has 8 GB RAM, but also 64-bit CPUs instead of the 32-bit we need (not sure if that's a blocker or not).

[gopherbot]

Change https://golang.org/cl/192332 mentions this issue: dashboard: skip x/tools on the linux-arm builder

[bradfitz]

The other possibility is to use go test -p 1 on the linux-arm builder.

That'd require a bit of coordinator changes.

[bcmills]

I'm not entirely sure that would even fix it: I don't know how much of the memory usage is due to parallelism vs. large test-cases, or how much of it ends up in long-running LSP servers.

[gopherbot]

Change https://golang.org/cl/192336 mentions this issue: go/packages,internal/lsp: skip memory-intensive tests on the linux-arm builder

[gopherbot]

Change https://golang.org/cl/192578 mentions this issue: all: skip more memory-intensive tests on linux-arm