cmd/go: 'go get' of module in GOPATH mode fails due to semantic version import

[nmiyake]

What version of Go are you using (go version)?

$ go version
go version go1.13.4 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Official golang:1.13.4 Docker image:

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build017263389=/tmp/go-build -gno-record-gcc-switches"

What did you do?

GO111MODULE=off go get github.com/mholt/archiver

(or a simple go get github.com/mholt/archiver in a GOPATH with no go.mod)

What did you expect to see?

Success: github.com/mholt/archiver and all of its dependencies should be fetched and exist in their proper locations within $GOPATH/src

What did you see instead?

package github.com/pierrec/lz4/v3: cannot find package "github.com/pierrec/lz4/v3" in any of:
	/usr/local/go/src/github.com/pierrec/lz4/v3 (from $GOROOT)
	/go/src/github.com/pierrec/lz4/v3 (from $GOPATH)

Details

github.com/mholt/archiver was recently updated to use Go modules. As part of this upgrade, one of its dependencies was upgraded to use github.com/pierrec/lz4/v3, which is another module. This module is at github.com/pierrec/lz4 (it does not explicitly create a "v3" directory).

It appears that there are clients that use "go get" in GOPATH mode to populate their GOPATH with their dependencies to build their projects in CI (presumably they are not vendoring). In this mode, "go get" can no longer be used to properly populate the GOPATH.

Setting GO111MODULE=on go get github.com/mholt/archiver causes the "get" operation to succeed, but is not helpful since it does not actually populate the GOPATH.

I believe that if the GOPATH were properly populated, the minimal module compatibility code would allow the project to build. However, there does not seem to be any way to use go get to populate the GOPATH properly in this scenario.

[jayconrod]

The issue here seems to be that github.com/pierrec/lz4 does not have a go.mod file on the master branch, which is what go get checks out by default. If I manually check out v3.1.0 in $GOPATH/src/github.com/pierrec/lz4, the command go get -d github.com/mholt/archiver succeeds.

Minimal module compatibility means that in GOPATH mode (GO111MODULE=on), a package may be imported without a major version suffix if:

• The importing package is not part of a module (there is no go.mod file in any parent directory).
• The imported package is part of a module (there is a go.mod file).
• The module path has a major version suffix (ends with /v2).
• The go.mod file is not in a major version subdirectory.

So if the file $GOPATH/src/github.com/pierrec/lz4/go.mod existed with the line module github.com/pierrec/lz4/v3, a package without a go.mod file would be able to import those packages without the /v3.

To be compatible with GOPATH mode, github.com/pierrec/lz4 could either add a go.mod to master, or they could move their v3 module into a v3 subdirectory. I don't think there's anything github.com/mholt/archiver needs to do differently here.

As a local workaround, you can either check out a specific tag of github.com/pierrec/lz4 or manually add a go.mod file.

Closing because this seems to be working as designed, and we probably won't change semantics of minimal module compatibility at this point.

[nmiyake]

Got it, all of this makes sense -- in trying to figure out what was going on, the piece I was missing/didn't properly consider was the "go.mod" file not being on the main branch. Agree that, given all of this, this is working as intended. Thanks for the explanation!