cmd/go: GO111MODULE=on and unset GOPATH generate incompatible module checksums

[jayschwa]

What version of Go are you using (go version)?

$ go version
go version go1.11.4 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jay/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/jay/spell"
GOPROXY=""
GORACE=""
GOROOT="/snap/go/3095"
GOTMPDIR=""
GOTOOLDIR="/snap/go/3095/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build944689084=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ rm go.sum
$ GO111MODULE=on go mod tidy
$ (unset GOPATH && go mod tidy)
go: verifying github.com/docker/docker@v0.7.3-0.20181128160009-852542b39767: checksum mismatch
	downloaded: h1:l6MPrA7HbTIeBptIanRxdS9gn7DhOjBUJr6gxO0ssKc=
	go.sum:     h1:KA+HbwLQPSH8VOiw6Pgn/7UP6s1LfkaxJO+48QIS/zk=

$ rm go.sum
$ (unset GOPATH && go mod tidy)
$ GO111MODULE=on go mod tidy
go: verifying github.com/docker/docker@v0.7.3-0.20181128160009-852542b39767: checksum mismatch
	downloaded: h1:KA+HbwLQPSH8VOiw6Pgn/7UP6s1LfkaxJO+48QIS/zk=
	go.sum:     h1:l6MPrA7HbTIeBptIanRxdS9gn7DhOjBUJr6gxO0ssKc=

What did you expect to see?

I expected the same (or at least compatible) go.sum to be generated

[julieqiu]

/cc @bcmills

[bcmills]

Dup of #29278. Run go clean -modcache and clear out the go.sum entries for any modules whose repos contain symlinks.

[jayschwa]

@bcmills, this is not a duplicate. The problem still occurs after go clean -modcache.

[bcmills]

Yes, it is a duplicate. Read the whole reply.

and clear out the go.sum entries for any modules whose repos contain symlinks.

~/src/github.com/docker/docker$ find ./* -type l
./integration-cli/fixtures/https/server-key.pem
./integration-cli/fixtures/https/client-cert.pem
./integration-cli/fixtures/https/server-cert.pem
./integration-cli/fixtures/https/client-key.pem
./integration-cli/fixtures/https/ca.pem

[jayschwa]

Read the whole bug report. The first step of my example is rm go.sum.

[bcmills]

Yes, and you ran unset GOPATH && go mod tidy. Did you also unset GOPATH when you ran go clean -modcache?

Here's what I get, starting from a clean GOPATH:

$ go1.11.4 mod init golang.org/issue/scratch
go: creating new go.mod: module golang.org/issue/scratch

$ cat >main.go <<EOF
package main
import _ "github.com/docker/docker/client"
func main() {}
EOF

$ go1.11.4 get -m github.com/docker/docker@v0.7.3-0.20181128160009-852542b39767
go: finding github.com/docker/docker v0.7.3-0.20181128160009-852542b39767

$ go1.11.4 mod tidy
go: downloading github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: finding github.com/docker/go-connections/tlsconfig latest
go: finding github.com/docker/go-connections/nat latest
go: finding github.com/docker/go-units v0.3.3
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding github.com/docker/distribution/reference latest
go: downloading github.com/opencontainers/go-digest v1.0.0-rc1
go: downloading github.com/docker/go-units v0.3.3
go: finding github.com/docker/go-connections v0.4.0
go: downloading github.com/pkg/errors v0.8.1
go: finding gotest.tools/env latest
go: finding gotest.tools/assert/cmp latest
go: finding gotest.tools/skip latest
go: finding gotest.tools/assert latest
go: finding github.com/sirupsen/logrus v1.3.0
go: finding github.com/gorilla/mux v1.6.2
go: downloading github.com/docker/go-connections v0.4.0
go: finding github.com/docker/go-connections/sockets latest
go: finding google.golang.org/grpc/codes latest
go: finding github.com/opencontainers/image-spec/specs-go latest
go: finding github.com/Azure/go-ansiterm/winterm latest
go: downloading github.com/sirupsen/logrus v1.3.0
go: downloading github.com/gorilla/mux v1.6.2
go: finding google.golang.org/grpc v1.17.0
go: finding gotest.tools v2.2.0+incompatible
go: downloading gotest.tools v2.2.0+incompatible
go: finding github.com/Azure/go-ansiterm latest
go: downloading github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: finding github.com/google/go-cmp/cmp latest
go: finding github.com/opencontainers/image-spec v1.0.1
go: finding github.com/google/go-cmp/cmp/cmpopts latest
go: finding github.com/gogo/protobuf/proto latest
go: finding github.com/Nvveen/Gotty latest
go: finding github.com/google/go-cmp v0.2.0
go: downloading github.com/opencontainers/image-spec v1.0.1
go: downloading github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
go: downloading google.golang.org/grpc v1.17.0
go: downloading github.com/google/go-cmp v0.2.0
go: finding golang.org/x/time/rate latest
go: finding golang.org/x/sys/unix latest
go: finding github.com/docker/distribution v2.7.0+incompatible
go: downloading github.com/docker/distribution v2.7.0+incompatible
go: finding github.com/gogo/protobuf v1.2.0
go: finding golang.org/x/time latest
go: downloading golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: finding golang.org/x/sys latest
go: downloading golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb
go: downloading github.com/gogo/protobuf v1.2.0
go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding github.com/golang/protobuf v1.2.0
go: finding github.com/golang/mock v1.1.1
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: finding google.golang.org/appengine v1.1.0
go: finding honnef.co/go/tools v0.0.0-20180728063816-88497007e858
go: finding cloud.google.com/go v0.26.0
go: finding golang.org/x/text v0.3.0
go: finding github.com/stretchr/testify v1.2.2
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3
go: finding github.com/client9/misspell v0.3.4
go: finding golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: finding golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52
go: finding github.com/kisielk/gotool v1.0.0
go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: finding golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33
go: finding github.com/stretchr/objx v0.1.1
go: downloading golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: downloading github.com/stretchr/testify v1.2.2
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: downloading golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: downloading google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: downloading github.com/golang/protobuf v1.2.0
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading github.com/davecgh/go-spew v1.1.1
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/Microsoft/go-winio v0.4.11
go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: downloading golang.org/x/text v0.3.0
go: downloading github.com/Microsoft/go-winio v0.4.11
go: downloading github.com/gorilla/context v1.1.1

$ GOPATH=$(mktemp -d) go1.11.4 mod tidy
go: finding github.com/Microsoft/go-winio v0.4.11
go: finding github.com/docker/go-units v0.3.3
go: finding github.com/docker/distribution v2.7.0+incompatible
go: finding github.com/docker/go-connections v0.4.0
go: finding github.com/google/go-cmp v0.2.0
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
go: finding github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: finding gotest.tools v2.2.0+incompatible
go: finding github.com/gorilla/mux v1.6.2
go: finding github.com/opencontainers/image-spec v1.0.1
go: finding github.com/gogo/protobuf v1.2.0
go: finding golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb
go: finding github.com/sirupsen/logrus v1.3.0
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: finding google.golang.org/grpc v1.17.0
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/stretchr/testify v1.2.2
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding github.com/stretchr/objx v0.1.1
go: finding golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33
go: finding github.com/golang/protobuf v1.2.0
go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: finding google.golang.org/appengine v1.1.0
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: finding honnef.co/go/tools v0.0.0-20180728063816-88497007e858
go: finding golang.org/x/text v0.3.0
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: finding github.com/client9/misspell v0.3.4
go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: finding github.com/kisielk/gotool v1.0.0
go: finding github.com/golang/mock v1.1.1
go: finding golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52
go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: finding cloud.google.com/go v0.26.0
go: downloading github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: extracting github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: downloading github.com/opencontainers/go-digest v1.0.0-rc1
go: downloading google.golang.org/grpc v1.17.0
go: downloading gotest.tools v2.2.0+incompatible
go: downloading github.com/docker/distribution v2.7.0+incompatible
go: downloading github.com/docker/go-connections v0.4.0
go: downloading github.com/pkg/errors v0.8.1
go: downloading github.com/gorilla/mux v1.6.2
go: extracting github.com/opencontainers/go-digest v1.0.0-rc1
go: extracting github.com/docker/go-connections v0.4.0
go: extracting github.com/pkg/errors v0.8.1
go: downloading github.com/Microsoft/go-winio v0.4.11
go: downloading golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: extracting github.com/gorilla/mux v1.6.2
go: extracting gotest.tools v2.2.0+incompatible
go: downloading github.com/gorilla/context v1.1.1
go: downloading github.com/docker/go-units v0.3.3
go: downloading github.com/sirupsen/logrus v1.3.0
go: downloading github.com/google/go-cmp v0.2.0
go: downloading github.com/opencontainers/image-spec v1.0.1
go: extracting github.com/docker/go-units v0.3.3
go: extracting github.com/gorilla/context v1.1.1
go: downloading github.com/stretchr/testify v1.2.2
go: extracting github.com/Microsoft/go-winio v0.4.11
go: downloading github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
go: downloading github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: extracting github.com/google/go-cmp v0.2.0
go: extracting github.com/sirupsen/logrus v1.3.0
go: downloading golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb
go: extracting github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
go: downloading github.com/gogo/protobuf v1.2.0
go: downloading golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: extracting github.com/stretchr/testify v1.2.2
go: extracting github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: downloading github.com/davecgh/go-spew v1.1.1
go: extracting github.com/opencontainers/image-spec v1.0.1
go: downloading github.com/pmezard/go-difflib v1.0.0
go: extracting github.com/pmezard/go-difflib v1.0.0
go: downloading github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: extracting github.com/davecgh/go-spew v1.1.1
go: downloading golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: extracting google.golang.org/grpc v1.17.0
go: extracting github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: extracting golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: downloading google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: extracting github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading github.com/golang/protobuf v1.2.0
go: extracting github.com/docker/distribution v2.7.0+incompatible
go: extracting golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: extracting golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: downloading golang.org/x/text v0.3.0
go: extracting github.com/golang/protobuf v1.2.0
go: downloading golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: extracting golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: extracting golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb
go: extracting github.com/gogo/protobuf v1.2.0
go: extracting google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: extracting golang.org/x/text v0.3.0

$ cat go.sum | grep docker/docker
github.com/docker/docker v0.7.3-0.20181128160009-852542b39767 h1:KA+HbwLQPSH8VOiw6Pgn/7UP6s1LfkaxJO+48QIS/zk=
github.com/docker/docker v0.7.3-0.20181128160009-852542b39767/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=


$ GOPATH=$(mktemp -d) go1.11.2 mod tidy
go: finding github.com/sirupsen/logrus v1.3.0
go: finding github.com/gorilla/mux v1.6.2
go: finding github.com/opencontainers/go-digest v1.0.0-rc1
go: finding github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5
go: finding github.com/docker/distribution v2.7.0+incompatible
go: finding github.com/google/go-cmp v0.2.0
go: finding github.com/Microsoft/go-winio v0.4.11
go: finding github.com/opencontainers/image-spec v1.0.1
go: finding gotest.tools v2.2.0+incompatible
go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
go: finding github.com/gorilla/context v1.1.1
go: finding github.com/pkg/errors v0.8.1
go: finding github.com/docker/go-units v0.3.3
go: finding github.com/docker/go-connections v0.4.0
go: finding github.com/stretchr/testify v1.2.2
go: finding golang.org/x/sys v0.0.0-20190109145017-48ac38b7c8cb
go: finding github.com/pmezard/go-difflib v1.0.0
go: finding github.com/konsorten/go-windows-terminal-sequences v1.0.1
go: finding github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: finding github.com/davecgh/go-spew v1.1.1
go: finding github.com/Azure/go-ansiterm v0.0.0-20170929234023-d6e3b3328b78
go: finding github.com/gogo/protobuf v1.2.0
go: finding golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33
go: finding golang.org/x/crypto v0.0.0-20180904163835-0709b304e793
go: finding google.golang.org/grpc v1.17.0
go: finding github.com/stretchr/objx v0.1.1
go: finding golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f
go: finding golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
go: finding golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3
go: finding google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8
go: finding honnef.co/go/tools v0.0.0-20180728063816-88497007e858
go: finding google.golang.org/appengine v1.1.0
go: finding github.com/client9/misspell v0.3.4
go: finding cloud.google.com/go v0.26.0
go: finding github.com/golang/mock v1.1.1
go: finding github.com/golang/protobuf v1.2.0
go: finding golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
go: finding github.com/kisielk/gotool v1.0.0
go: finding golang.org/x/sys v0.0.0-20180830151530-49385e6e1522
go: finding golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52
go: finding golang.org/x/text v0.3.0
go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
go: downloading github.com/docker/docker v0.7.3-0.20181128160009-852542b39767
go: verifying github.com/docker/docker@v0.7.3-0.20181128160009-852542b39767: checksum mismatch
        downloaded: h1:l6MPrA7HbTIeBptIanRxdS9gn7DhOjBUJr6gxO0ssKc=
        go.sum:     h1:KA+HbwLQPSH8VOiw6Pgn/7UP6s1LfkaxJO+48QIS/zk=

The error in your snippet is telling you exactly what is wrong: the module cache in your default GOPATH (that is, $USER/go/pkg/mod) has an incorrect (pre-1.11.4) copy of the module, while the cache in your actual GOPATH ($GOPATH/pkg/mod) has the correct one.

[jayschwa]

Did you also unset GOPATH when you ran go clean -modcache?

No, that explains it. Sorry for the bother (and salt).