cmd/go: tidy and vendor do not record dependencies of '// +build ignore' go:generate sources

[realnedsanders]

What version of Go are you using (go version)?

$ go version
go version go1.14 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/aescaler/.cache/go-build"
GOENV="/home/aescaler/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/aescaler/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build295363184=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Create a package with dependencies built from go:generate tags

//go:generate go run generator.go

Attempt to use go mod vendor and go mod tidy

https://github.com/angelofdeauth/xnotify/

under pkg/box:

generate.go script called using preprocessor tags in box.go

Since generate.go is not using package box, go mod tidy and go mod vendor do not correctly resolve its imports.

What did you expect to see?

go mod vendor and go mod tidy correctly resolve imports of build-time dependencies.

What did you see instead?

imports are not correctly resolved, added to go.mod or vendor/modules.txt, or downloaded to the vendor folder.

[seankhliao]

Does go throw an error? Is there a specific import that should be added but isn't?

[bcmills]

You are using the word “correctly”, but it isn't at all obvious to me that the observed behavior is “incorrect”. Can you describe what you expected to see and what you did see in terms of the actual dependencies used?

[realnedsanders]

Does go throw an error? Is there a specific import that should be added but isn't?

You are using the word “correctly”, but it isn't at all obvious to me that the observed behavior is “incorrect”. Can you describe what you expected to see and what you did see in terms of the actual dependencies used?

https://github.com/angelofdeauth/xnotify/blob/4263fb794911e4bf6e6350ff262b593ae07843b6/pkg/box/generator.go is a script that is run at build time using go generate, and is called from https://github.com/angelofdeauth/xnotify/blob/4263fb794911e4bf6e6350ff262b593ae07843b6/pkg/box/box.go#L9 via the //go:generate preprocessor tags.
generator.go uses package main as it isn't compiled as part of the module/package and is instead just a one-off which generates a file that is included in the module/package.

Without this line: https://github.com/angelofdeauth/xnotify/blob/4263fb794911e4bf6e6350ff262b593ae07843b6/pkg/box/hack.go#L11 , go mod vendor and go mod tidy do not recognize "github.com/gobuffalo/here" as a project dependency at all, and there is no dependency listed in my go.mod, vendor/modules.txt or files in vendor/{{importname}}.
This is a problem, as when I try to run go generate, it fails stating that the dependency is not found in the current module.

[realnedsanders]

I just realized, it's actually the //+build ignore directive in generator.go https://github.com/angelofdeauth/xnotify/blob/4263fb794911e4bf6e6350ff262b593ae07843b6/pkg/box/generator.go#L9 that's causing this. I would argue the intended behaviour here is still valid, as generator.go should not be built into the final executable, but the tooling functionality does not reflect that ability.

[bcmills]

If you don't want the file to be ignored, use a build constraint other than ignore.

(// +build tools is common: see http://golang.org/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module.)

[bcmills]

Or, I suppose, if you want the dependency to be resolved dynamically instead of a specific version, set GOFLAGS=-mod=mod or pass -mod=mod explicitly on the go:generate command line, so that go run doesn't try to find the dependency in the local vendor directory.

[bcmills]

Duplicate of #29598

[realnedsanders]

Awesome, thanks for the link @bcmills . I've looked at the docs on golang.org, they're less descriptive. This is much more clear.