cmd/go: revise 'go help' documentation for modules

Module-related help pages now contain a brief summary and point to the
reference documentation at golang.org/ref/mod for details.

Help pages for commands like 'go get' still describe the basic usage
and summarize flags but don't provide as much background detail.

Fixes #41427
Fixes #43419

Change-Id: Icacd38e0f33c352c447cc5a496c99674493abde2
Reviewed-on: https://go-review.googlesource.com/c/go/+/282615
Trust: Jay Conrod <[email protected]>
Run-TryBot: Jay Conrod <[email protected]>
TryBot-Result: Go Bot <[email protected]>
Reviewed-by: Bryan C. Mills <[email protected]>

Author: Jay Conrod

src/cmd/go/alldocs.go

@@ -266,7 +266,7 @@ listed in the GOPATH environment variable.
 (See 'go help gopath-get' and 'go help gopath'.)
 
 When using modules, downloaded packages are stored in the module cache.
-(See 'go help module-get' and 'go help goproxy'.)
+See https://golang.org/ref/mod#module-cache.
 
 When using modules, an additional variant of the go-import meta tag is
 recognized and is preferred over those listing version control systems.
@@ -276,7 +276,8 @@ That variant uses "mod" as the vcs in the content value, as in:
 
 This tag means to fetch modules with paths beginning with example.org
 from the module proxy available at the URL https://code.org/moduleproxy.
-See 'go help goproxy' for details about the proxy protocol.
+See https://golang.org/ref/mod#goproxy-protocol for details about the
+proxy protocol.
 
 Import path checking
 
@@ -483,6 +484,10 @@ See 'go help env' for details.
 
 General-purpose environment variables:
 
+	GO111MODULE
+		Controls whether the go command runs in module-aware mode or GOPATH mode.
+		May be "off", "on", or "auto".
+		See https://golang.org/ref/mod#mod-commands.
 	GCCGO
 		The gccgo command to run for 'go build -compiler=gccgo'.
 	GOARCH
@@ -521,20 +526,24 @@ General-purpose environment variables:
 	GOPATH
 		For more details see: 'go help gopath'.
 	GOPROXY
-		URL of Go module proxy. See 'go help modules'.
+		URL of Go module proxy. See https://golang.org/ref/mod#environment-variables
+		and https://golang.org/ref/mod#module-proxy for details.
 	GOPRIVATE, GONOPROXY, GONOSUMDB
 		Comma-separated list of glob patterns (in the syntax of Go's path.Match)
 		of module path prefixes that should always be fetched directly
 		or that should not be compared against the checksum database.
-		See 'go help private'.
+		See https://golang.org/ref/mod#private-modules.
 	GOROOT
 		The root of the go tree.
 	GOSUMDB
 		The name of checksum database to use and optionally its public key and
-		URL. See 'go help module-auth'.
+		URL. See https://golang.org/ref/mod#authenticating.
 	GOTMPDIR
 		The directory where the go command will write
 		temporary source files, packages, and binaries.
+	GOVCS
+	  Lists version control commands that may be used with matching servers.
+		See 'go help vcs'.
 
 Environment variables for use with cgo:
 

src/cmd/go/internal/help/helpdoc.go

@@ -304,7 +304,7 @@ For more about build flags, see 'go help build'.
 
 For more about specifying packages, see 'go help packages'.
 
-For more about modules, see 'go help modules'.
+For more about modules, see https://golang.org/ref/mod.
 	`,
 }
 

src/cmd/go/internal/list/list.go

@@ -52,7 +52,9 @@ corresponding to this Go struct:
 
 The -x flag causes download to print the commands download executes.
 
-See 'go help modules' for more about module queries.
+See https://golang.org/ref/mod#go-mod-download for more about 'go mod download'.
+
+See https://golang.org/ref/mod#version-queries for more about version queries.
 	`,
 }
 

src/cmd/go/internal/modcmd/download.go

@@ -122,9 +122,7 @@ Note that this only describes the go.mod file itself, not other modules
 referred to indirectly. For the full set of modules available to a build,
 use 'go list -m -json all'.
 
-For example, a tool can obtain the go.mod as a data structure by
-parsing the output of 'go mod edit -json' and can then make changes
-by invoking 'go mod edit' with -require, -exclude, and so on.
+See https://golang.org/ref/mod#go-mod-edit for more about 'go mod edit'.
 	`,
 }
 

src/cmd/go/internal/modcmd/edit.go

@@ -26,6 +26,8 @@ Graph prints the module requirement graph (with replacements applied)
 in text form. Each line in the output has two space-separated fields: a module
 and one of its requirements. Each module is identified as a string of the form
 path@version, except for the main module, which has no @version suffix.
+
+See https://golang.org/ref/mod#go-mod-graph for more about 'go mod graph'.
 	`,
 	Run: runGraph,
 }

src/cmd/go/internal/modcmd/graph.go

@@ -27,6 +27,8 @@ Gopkg.lock), and the current directory (if in GOPATH).
 
 If a configuration file for a vendoring tool is present, init will attempt to
 import module requirements from it.
+
+See https://golang.org/ref/mod#go-mod-init for more about 'go mod init'.
 `,
 	Run: runInit,
 }

src/cmd/go/internal/modcmd/init.go

@@ -29,6 +29,8 @@ to standard error.
 
 The -e flag causes tidy to attempt to proceed despite errors
 encountered while loading packages.
+
+See https://golang.org/ref/mod#go-mod-tidy for more about 'go mod tidy'.
 	`,
 	Run: runTidy,
 }

src/cmd/go/internal/modcmd/tidy.go

@@ -38,6 +38,8 @@ modules and packages to standard error.
 
 The -e flag causes vendor to attempt to proceed despite errors
 encountered while loading packages.
+
+See https://golang.org/ref/mod#go-mod-vendor for more about 'go mod vendor'.
 	`,
 	Run: runVendor,
 }

src/cmd/go/internal/modcmd/vendor.go

@@ -31,6 +31,8 @@ modified since being downloaded. If all the modules are unmodified,
 verify prints "all modules verified." Otherwise it reports which
 modules have been changed and causes 'go mod' to exit with a
 non-zero status.
+
+See https://golang.org/ref/mod#go-mod-verify for more about 'go mod verify'.
 	`,
 	Run: runVerify,
 }

src/cmd/go/internal/modcmd/verify.go

@@ -48,6 +48,8 @@ For example:
 	# golang.org/x/text/encoding
 	(main module does not need package golang.org/x/text/encoding)
 	$
+
+See https://golang.org/ref/mod#go-mod-why for more about 'go mod why'.
 	`,
 }
 

src/cmd/go/internal/modcmd/why.go

@@ -768,90 +768,14 @@ var HelpModuleAuth = &base.Command{
 	UsageLine: "module-auth",
 	Short:     "module authentication using go.sum",
 	Long: `
-The go command tries to authenticate every downloaded module,
-checking that the bits downloaded for a specific module version today
-match bits downloaded yesterday. This ensures repeatable builds
-and detects introduction of unexpected changes, malicious or not.
-
-In each module's root, alongside go.mod, the go command maintains
-a file named go.sum containing the cryptographic checksums of the
-module's dependencies.
-
-The form of each line in go.sum is three fields:
-
-	<module> <version>[/go.mod] <hash>
-
-Each known module version results in two lines in the go.sum file.
-The first line gives the hash of the module version's file tree.
-The second line appends "/go.mod" to the version and gives the hash
-of only the module version's (possibly synthesized) go.mod file.
-The go.mod-only hash allows downloading and authenticating a
-module version's go.mod file, which is needed to compute the
-dependency graph, without also downloading all the module's source code.
-
-The hash begins with an algorithm prefix of the form "h<N>:".
-The only defined algorithm prefix is "h1:", which uses SHA-256.
-
-Module authentication failures
-
-The go command maintains a cache of downloaded packages and computes
-and records the cryptographic checksum of each package at download time.
-In normal operation, the go command checks the main module's go.sum file
-against these precomputed checksums instead of recomputing them on
-each command invocation. The 'go mod verify' command checks that
-the cached copies of module downloads still match both their recorded
-checksums and the entries in go.sum.
-
-In day-to-day development, the checksum of a given module version
-should never change. Each time a dependency is used by a given main
-module, the go command checks its local cached copy, freshly
-downloaded or not, against the main module's go.sum. If the checksums
-don't match, the go command reports the mismatch as a security error
-and refuses to run the build. When this happens, proceed with caution:
-code changing unexpectedly means today's build will not match
-yesterday's, and the unexpected change may not be beneficial.
-
-If the go command reports a mismatch in go.sum, the downloaded code
-for the reported module version does not match the one used in a
-previous build of the main module. It is important at that point
-to find out what the right checksum should be, to decide whether
-go.sum is wrong or the downloaded code is wrong. Usually go.sum is right:
-you want to use the same code you used yesterday.
-
-If a downloaded module is not yet included in go.sum and it is a publicly
-available module, the go command consults the Go checksum database to fetch
-the expected go.sum lines. If the downloaded code does not match those
-lines, the go command reports the mismatch and exits. Note that the
-database is not consulted for module versions already listed in go.sum.
-
-If a go.sum mismatch is reported, it is always worth investigating why
-the code downloaded today differs from what was downloaded yesterday.
-
-The GOSUMDB environment variable identifies the name of checksum database
-to use and optionally its public key and URL, as in:
-
-	GOSUMDB="sum.golang.org"
-	GOSUMDB="sum.golang.org+<publickey>"
-	GOSUMDB="sum.golang.org+<publickey> https://sum.golang.org"
-
-The go command knows the public key of sum.golang.org, and also that the name
-sum.golang.google.cn (available inside mainland China) connects to the
-sum.golang.org checksum database; use of any other database requires giving
-the public key explicitly.
-The URL defaults to "https://" followed by the database name.
-
-GOSUMDB defaults to "sum.golang.org", the Go checksum database run by Google.
-See https://sum.golang.org/privacy for the service's privacy policy.
-
-If GOSUMDB is set to "off", or if "go get" is invoked with the -insecure flag,
-the checksum database is not consulted, and all unrecognized modules are
-accepted, at the cost of giving up the security guarantee of verified repeatable
-downloads for all modules. A better way to bypass the checksum database
-for specific modules is to use the GOPRIVATE or GONOSUMDB environment
-variables. See 'go help private' for details.
-
-The 'go env -w' command (see 'go help env') can be used to set these variables
-for future go command invocations.
+When the go command downloads a module zip file or go.mod file into the
+module cache, it computes a cryptographic hash and compares it with a known
+value to verify the file hasn't changed since it was first downloaded. Known
+hashes are stored in a file in the module root directory named go.sum. Hashes
+may also be downloaded from the checksum database depending on the values of
+GOSUMDB, GOPRIVATE, and GONOSUMDB.
+
+For details, see https://golang.org/ref/mod#authenticating.
 `,
 }
 
@@ -865,8 +789,8 @@ regardless of source, against the public Go checksum database at sum.golang.org.
 These defaults work well for publicly available source code.
 
 The GOPRIVATE environment variable controls which modules the go command
-considers to be private (not available publicly) and should therefore not use the
-proxy or checksum database. The variable is a comma-separated list of
+considers to be private (not available publicly) and should therefore not use
+the proxy or checksum database. The variable is a comma-separated list of
 glob patterns (in the syntax of Go's path.Match) of module path prefixes.
 For example,
 
@@ -876,10 +800,6 @@ causes the go command to treat as private any module with a path prefix
 matching either pattern, including git.corp.example.com/xyzzy, rsc.io/private,
 and rsc.io/private/quux.
 
-The GOPRIVATE environment variable may be used by other tools as well to
-identify non-public modules. For example, an editor could use GOPRIVATE
-to decide whether to hyperlink a package import to a godoc.org page.
-
 For fine-grained control over module download and validation, the GONOPROXY
 and GONOSUMDB environment variables accept the same kind of glob list
 and override GOPRIVATE for the specific decision of whether to use the proxy
@@ -892,18 +812,14 @@ users would configure go using:
 	GOPROXY=proxy.example.com
 	GONOPROXY=none
 
-This would tell the go command and other tools that modules beginning with
-a corp.example.com subdomain are private but that the company proxy should
-be used for downloading both public and private modules, because
-GONOPROXY has been set to a pattern that won't match any modules,
-overriding GOPRIVATE.
-
 The GOPRIVATE variable is also used to define the "public" and "private"
 patterns for the GOVCS variable; see 'go help vcs'. For that usage,
 GOPRIVATE applies even in GOPATH mode. In that case, it matches import paths
 instead of module paths.
 
 The 'go env -w' command (see 'go help env') can be used to set these variables
 for future go command invocations.
+
+For more details, see https://golang.org/ref/mod#private-modules.
 `,
 }

src/cmd/go/internal/modfetch/fetch.go

@@ -36,65 +36,8 @@ URLs of a specified form. The requests have no query parameters, so even
 a site serving from a fixed file system (including a file:/// URL)
 can be a module proxy.
 
-The GET requests sent to a Go module proxy are:
-
-GET $GOPROXY/<module>/@v/list returns a list of known versions of the given
-module, one per line.
-
-GET $GOPROXY/<module>/@v/<version>.info returns JSON-formatted metadata
-about that version of the given module.
-
-GET $GOPROXY/<module>/@v/<version>.mod returns the go.mod file
-for that version of the given module.
-
-GET $GOPROXY/<module>/@v/<version>.zip returns the zip archive
-for that version of the given module.
-
-GET $GOPROXY/<module>/@latest returns JSON-formatted metadata about the
-latest known version of the given module in the same format as
-<module>/@v/<version>.info. The latest version should be the version of
-the module the go command may use if <module>/@v/list is empty or no
-listed version is suitable. <module>/@latest is optional and may not
-be implemented by a module proxy.
-
-When resolving the latest version of a module, the go command will request
-<module>/@v/list, then, if no suitable versions are found, <module>/@latest.
-The go command prefers, in order: the semantically highest release version,
-the semantically highest pre-release version, and the chronologically
-most recent pseudo-version. In Go 1.12 and earlier, the go command considered
-pseudo-versions in <module>/@v/list to be pre-release versions, but this is
-no longer true since Go 1.13.
-
-To avoid problems when serving from case-sensitive file systems,
-the <module> and <version> elements are case-encoded, replacing every
-uppercase letter with an exclamation mark followed by the corresponding
-lower-case letter: github.com/Azure encodes as github.com/!azure.
-
-The JSON-formatted metadata about a given module corresponds to
-this Go data structure, which may be expanded in the future:
-
-    type Info struct {
-        Version string    // version string
-        Time    time.Time // commit time
-    }
-
-The zip archive for a specific version of a given module is a
-standard zip file that contains the file tree corresponding
-to the module's source code and related files. The archive uses
-slash-separated paths, and every file path in the archive must
-begin with <module>@<version>/, where the module and version are
-substituted directly, not case-encoded. The root of the module
-file tree corresponds to the <module>@<version>/ prefix in the
-archive.
-
-Even when downloading directly from version control systems,
-the go command synthesizes explicit info, mod, and zip files
-and stores them in its local cache, $GOPATH/pkg/mod/cache/download,
-the same as if it had downloaded them directly from a proxy.
-The cache layout is the same as the proxy URL space, so
-serving $GOPATH/pkg/mod/cache/download at (or copying it to)
-https://example.com/proxy would let other users access those
-cached module versions with GOPROXY=https://example.com/proxy.
+For details on the GOPROXY protocol, see
+https://golang.org/ref/mod#goproxy-protocol.
 `,
 }