proposal: testing: fail if the same test function is defined in both the _test and non-_test packages

[bcmills]

(This proposal prompted by https://go-review.googlesource.com/c/go/+/488855/10//COMMIT_MSG#22.)

If a Go package has both an internal test (package example) and an external test (package example_test), the resulting test binary runs the Test, Fuzz, Example, and Benchmark functions from both packages.

If a TestMain function is defined in both packages, test setup fails with multiple definitions of TestMain.

However, if other testing functions are defined in both packages, they are both run. That leads to a number of confusing effects:

1. The same test name can be shown with two different results (say, PASS and SKIP, or PASS and FAIL), even if the test flags only request to run each package once. That may confuse tools that ingest test results.
2. If the tests run in parallel, logs and JSON events for the two tests may be interleaved, with no way to distinguish the two. (t.Log messages will list the correct file and line number, but log messages originating in helper functions may be completely ambiguous.)
3. It is impossible to use the -run flag to run only one of the two tests but not the other.

For the above reasons, I propose that a test binary that defines two different Test, Fuzz, Example, or Benchmark functions with the same name should fail at run-time.

(A GODEBUG setting should be used to restrict this failure to only tests whose go version is at least the release in which this change is introduced.)

[bcmills]

Alternately, this could be implemented as a vet check, included in the subset of vet checks run by default during go test.

However, I prefer the testing approach: the cmd/vet alternative seems more difficult to implement (especially in a way that complies with #56986), and either way it would cause go test of the package to fail.

[ianlancetaylor]

Why fail at runtime rather than failing when cmd/go generates the _testmain.go file?

[bcmills]

Failing at run time allows a user to revert to the prior behavior easily using a //go:debug directive. Failing at generation time would make it a hard break for any existing duplicated tests when updating the go version.

[KianYang-Lee]

If a TestMain function is defined in both packages, test setup fails with multiple definitions of TestMain.

Why would this be the case? Aren't external and internal packages are considered as different packages, and so should have no issue with name clashing?