cmd/go: toolchain directive in go.mod being updated unnecessarily

[dprotaso]

There doesn't seem to be a way to stop the toolchain directive from being updated when running various go commands

In our OSS project we rely on the go directive in the go.mod to ensure a min go version. We do not want to use the toolchain directive as we always expect it to match the go directive.

some user feedback

• https://gophers.slack.com/archives/C0VPK4Z5E/p1704471476388119
• https://gophers.slack.com/archives/C9BMAAFFB/p1708525315637839

Go Version

go version go1.22.0 darwin/amd64

Go ENV

expand

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/Users/dprotasowski/Library/Caches/go-build'
GOENV='/Users/dprotasowski/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/dprotasowski/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/dprotasowski/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/Cellar/go/1.22.0/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/Cellar/go/1.22.0/libexec/pkg/tool/darwin_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/Users/dprotasowski/work/test/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/xs/xfp1z4cn643c46w69lv8bv8w0000gp/T/go-build4163821748=/tmp/go-build -gno-record-gcc-switches -fno-common'

Scenario 1 - creating a new module and downgrading go

What did you do

$ mkdir test && cd test
$ go mod init blah
$ go get go@1.21

What did you see happen?

• Tool chain directive of 1.22.0 was added - but this isn't necessary
• min go version set to 1.21.7

What did you expect to happen?

• No toolchain directive to be added
• Min go version set to 1.21 - since I didn't specify the point release

Scenario 2 - pulling a new dependency that upgraded the go version

What did you do

Project structure

--- go.mod
module blah

go 1.19

---- main.go
package main

import "fmt"

func main() {
	fmt.Println("vim-go")
}

Fetch a dependency that updates the go min version

$ go get knative.dev/pkg@9f033a7

What did you see happen?

console output

go: upgraded go 1.19 => 1.21
go: added toolchain go1.22.0
go: added knative.dev/pkg v0.0.0-20240221065059-9f033a7b77f7

go.mod output

module blah

go 1.21

toolchain go1.22.0

require knative.dev/pkg v0.0.0-20240221065059-9f033a7b77f7 // indirect

What did you expect to happen?

toolchain directive shouldn't appear. We want it to always default to the same as the go directive

[bcmills]

Please fill out the complete issue template.

In particular: what version of Go are you using, what was your go directive set to initially, and what commands did you run that added the toolchain directive?

[dprotaso]

@bcmills updated - can you take a look again and see if I'm missing anything?

[dylan-bourque]

Adding my own experiences for reference.

At work, we would definitely prefer to have the ability to disable the automatic toolchain upgrades (and possibly fail builds that would otherwise require a newer toolchain). We have many modules with the go directive in their go.mod files targeting Go versions from 1.16 to 1.21. We also do not proactively update those directives across the codebase when a new Go version is released.

This feature started causing issues for us when one engineer using Go 1.21.x locally updated some dependency and ran go mod tidy, which silently updated the go directive in that module and added the new toolchain one. We were all surprised when this change failed to build in CI, which was still using Go 1.20.x.

Without the ability to opt out of automatic toolchain upgrades we are now forced to synchronize the Go version across CI and every developer machine. While that's not an unreasonable stance in the general case, it is a new restriction/requirement that, as far as I've seen, is not called out in the release notes.

We have tried setting GOTOOLCHAIN to local but that doesn't seem to affect the behavior of go mod tidy. It will still upgrade both the go and toolchain directives.

One could also argue that forcing more conservative development shops to always stay on the most current Go release, despite the previous minor release being explicitly supported, is a bad situation.

[bcmills]

As of Go 1.21, the initial toolchain release has a .0 suffix (see #57631), so go 1.21 is a development version of the language — not a released version.

And for the toolchain directive in particular, per https://go.dev/doc/toolchain: “For repeatability, any command that updates the go line also updates the toolchain line to record its own toolchain name.”

Some specifics:

• go get knative.dev/pkg@9f033a7 is adding the toolchain line because it is upgrading the go line. That seems to be working as designed.

• go get go@1.21 is resolving to a specific point release because that's what it usually does for version prefixes. Unfortunately, the language version is a prefix of the release version, and we don't have a good way to force the language version instead of resolving the prefix. That probably needs some more thought.

• I don't understand why go get go@1.21 is not writing out a toolchain line. That is at the very least a mismatch between the documentation and the implementation.

[bcmills]

At work, we would definitely prefer to have the ability to disable the automatic toolchain upgrades (and possibly fail builds that would otherwise require a newer toolchain).

You do have that ability: you can set GOTOOLCHAIN=local in your process environment or GOENV file. (You can even edit $GOROOT/go.env to make that the default for your Go toolchain installation, if you are so inclined.)

You can also force go get to stick to a particular go version, the same way you would for any other dependency that you want to avoid upgrading, by passing that version as an explicit argument to go get: go get go@1.20 will downgrade to go 1.20, and go get knative.dev/pkg@9f033a7 go@1.20 will correctly error out (and report that those versions are not mutually compatible).

Without the ability to opt out of automatic toolchain upgrades we are now forced to synchronize the Go version across CI and every developer machine.

That is exactly the problem that GOTOOLCHAIN=auto is supposed to mitigate: when everyone is on Go 1.21.0 or above, the developers' machines should automatically download whatever toolchain is needed to work in the module.

[dylan-bourque]

You do have that ability: you can set GOTOOLCHAIN=local in your process environment

This works, but it also means that as soon as anyone moves to a newer Go release then everyone else in the organization must also move to that same newer version immediately. This is a problem in tightly regulated environments where, as an example Go 1.22.x may not be "certified" but Go 1.21.5 is. As it stands today it's not possible to both fix CI to 1.21.x and also allow developers to use a newer 1.21.y or 1.22.

You can also force go get to stick to a particular go version ... by passing that version as an explicit argument to go get: go get go@1.20

TIL. Thanks for this.

[cevich]

you can set GOTOOLCHAIN=local in your process environment

There are environments where setting an env. var. is not possible. For example, the public Renovate service does not support this. There are other ways to pin the golang version in Renovate, but they're undesirable for maintainability reasons. Devcontainers could be another example, where the tooling env. is shared, and must remain static, for consistency across a whole team of developers. In any case, my point is there are places where setting env. vars. isn't a viable workaround. IMHO such major behavior changes should almost always default to the previous or "least disruptive" option.