Skip to content

cmd/go: go list with -reuse fails to invalidate @latest when the previous result was an error #61415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
bcmills opened this issue Jul 18, 2023 · 3 comments
Closed

cmd/go: go list with -reuse fails to invalidate @latest when the previous result was an error #61415

bcmills opened this issue Jul 18, 2023 · 3 comments
Assignees
@bcmills
Labels
FrozenDueToAge GoCommand cmd/go modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.22

Comments

@bcmills
Copy link
Contributor

bcmills commented Jul 18, 2023

What version of Go are you using (go version)?

$ go version
go version devel go1.21-18e17e2c Thu Jun 29 23:06:46 2023 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/usr/local/google/home/bcmills/.cache/go-build'
GOENV='/usr/local/google/home/bcmills/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/tmp/tmp.n0kPM4RTS6/.gopath'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/google/home/bcmills/sdk/gotip'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/google/home/bcmills/sdk/gotip/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='devel go1.21-18e17e2c Thu Jun 29 23:06:46 2023 +0000'
GCCGO='/usr/bin/gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2124346309=/tmp/go-build -gno-record-gcc-switches'

What did you do?

$ cat > reuse.json
{
        "Path": "go.dkinom.dev/baseconv/characters",
        "Version": "v0.0.0-20230716190303-357f22feb6b8",
        "Query": "latest",
        "Origin": {
                "VCS": "git",
                "URL": "https://github.com/godsfood/baseconv",
                "Subdir": "characters",
                "Hash": "357f22feb6b8d456266ad5ce2aa31fd042d44de2"
        }
}

$ GOPROXY=direct go list -m -retracted -e -x -v -json=Version,Versions,Error,Path,Query,Origin,Reuse --versions -reuse=reuse.json -- go.dkinom.dev/baseconv/characters@latest

What did you expect to see?

$ GOPROXY=direct go list -m -retracted -e -x -v -json=Version,Versions,Error,Path,Query,Origin,Reuse --versions -- go.dkinom.dev/baseconv/characters@latest
# get https://go.dkinom.dev/baseconv/characters?go-get=1
# get https://go.dkinom.dev/baseconv/characters?go-get=1: 200 OK (0.081s)
get "go.dkinom.dev/baseconv/characters": found meta tag vcs.metaImport{Prefix:"go.dkinom.dev/baseconv", VCS:"git", RepoRoot:"https://github.com/godsfood/baseconv"} at //go.dkinom.dev/baseconv/characters?go-get=1
get "go.dkinom.dev/baseconv/characters": verifying non-authoritative meta tag
# get https://go.dkinom.dev/baseconv?go-get=1
# get https://go.dkinom.dev/baseconv?go-get=1: 200 OK (0.040s)
mkdir -p /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs # git3 https://github.com/godsfood/baseconv
# lock /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949.lock
# /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949 for git3 https://github.com/godsfood/baseconv
cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git ls-remote -q origin
0.181s # cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git ls-remote -q origin
cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 3bd85f839063b2b23ccab1be6cd4c87940d2cda1 --
0.008s # cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 3bd85f839063b2b23ccab1be6cd4c87940d2cda1 --
cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git for-each-ref --format %(refname) refs/tags --merged 3bd85f839063b2b23ccab1be6cd4c87940d2cda1
0.004s # cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git for-each-ref --format %(refname) refs/tags --merged 3bd85f839063b2b23ccab1be6cd4c87940d2cda1
cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 3bd85f839063 --
0.007s # cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git -c log.showsignature=false log --no-decorate -n1 '--format=format:%H %ct %D' 3bd85f839063 --
cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git cat-file blob 3bd85f839063b2b23ccab1be6cd4c87940d2cda1:characters/go.mod
0.004s # cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git cat-file blob 3bd85f839063b2b23ccab1be6cd4c87940d2cda1:characters/go.mod
{
        "Path": "go.dkinom.dev/baseconv/characters",
        "Version": "v0.0.0-20230718102946-3bd85f839063",
        "Query": "latest",
        "Origin": {
                "VCS": "git",
                "URL": "https://github.com/godsfood/baseconv",
                "Subdir": "characters",
                "TagPrefix": "characters/",
                "TagSum": "t1:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=",
                "Ref": "HEAD",
                "Hash": "3bd85f839063b2b23ccab1be6cd4c87940d2cda1"
        }
}

What did you see instead?

$ GOPROXY=direct go list -m -retracted -e -x -v -json=Version,Versions,Error,Path,Query,Origin,Reuse --versions -reuse=reuse.json -- go.dkinom.dev/baseconv/characters@latest
# get https://go.dkinom.dev/baseconv/characters?go-get=1
# get https://go.dkinom.dev/baseconv/characters?go-get=1: 200 OK (0.073s)
get "go.dkinom.dev/baseconv/characters": found meta tag vcs.metaImport{Prefix:"go.dkinom.dev/baseconv", VCS:"git", RepoRoot:"https://github.com/godsfood/baseconv"} at //go.dkinom.dev/baseconv/characters?go-get=1
get "go.dkinom.dev/baseconv/characters": verifying non-authoritative meta tag
# get https://go.dkinom.dev/baseconv?go-get=1
# get https://go.dkinom.dev/baseconv?go-get=1: 200 OK (0.026s)
mkdir -p /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs # git3 https://github.com/godsfood/baseconv
# lock /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949.lock
# /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949 for git3 https://github.com/godsfood/baseconv
cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git ls-remote -q origin
0.198s # cd /tmp/tmp.n0kPM4RTS6/.gopath/pkg/mod/cache/vcs/7b8cca26b47a7a5711e2adacad45276205192931f68c3f0673d760c865d00949; git ls-remote -q origin
{
        "Path": "go.dkinom.dev/baseconv/characters",
        "Version": "v0.0.0-20230716190303-357f22feb6b8",
        "Query": "latest",
        "Origin": {
                "VCS": "git",
                "URL": "https://github.com/godsfood/baseconv",
                "Subdir": "characters",
                "Hash": "357f22feb6b8d456266ad5ce2aa31fd042d44de2"
        },
        "Reuse": true
}
@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go modules labels Jul 18, 2023
@bcmills bcmills added this to the Go1.22 milestone Jul 18, 2023
@bcmills bcmills self-assigned this Jul 18, 2023
@bcmills bcmills mentioned this issue Jul 18, 2023
Closed
@bcmills bcmills changed the title cmd/go: 'go list' with '-reuse' fails to invalidate @latest cmd/go: go list with -reuse fails to invalidate @latest Jul 18, 2023
@bcmills bcmills changed the title cmd/go: go list with -reuse fails to invalidate @latest cmd/go: go list with -reuse fails to invalidate @latest when the previous result was an error Jul 18, 2023
@bcmills
Copy link
Contributor Author

bcmills commented Jul 18, 2023

Here's the problem. When the @latest query results in an error (as it would have for godsfood/baseconv@8514a09), the Origin information for the error does not include the Ref that failed to resolve.

When we try to -reuse that origin again, it doesn't check that Ref still fails to resolve, and so it doesn't invalidate the result as it should.

$ GOPROXY=direct go list -json -m --versions -e -reuse=latest.json github.com/bcmills/issue61415/nested@latest
{
        "Path": "github.com/bcmills/issue61415/nested",
        "Version": "latest",
        "Error": {
                "Err": "module github.com/bcmills/issue61415/nested: no matching versions for query \"latest\""
        },
        "Origin": {
                "VCS": "git",
                "URL": "https://github.com/bcmills/issue61415",
                "Subdir": "nested",
                "TagPrefix": "nested/",
                "TagSum": "t1:47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU="
        }
}

@bcmills bcmills added the NeedsFix The path to resolution is known, but the work has not been done. label Jul 18, 2023
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 18, 2023
@bcmills bcmills mentioned this issue Jul 18, 2023
Closed
@bcmills bcmills mentioned this issue Sep 19, 2023
Closed
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/542717 mentions this issue: cmd/go: propagate origin information for inexact module queries

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/543216 mentions this issue: cmd/go: use a nil Origin to represent "unchackable"

gopherbot pushed a commit that referenced this issue Nov 17, 2023
@gopherbot
cmd/go: use a nil *Origin to represent "uncheckable"
0b90c7d 
Previously, we used the presence of individual origin fields
to decide whether an Origin could be checked for staleness,
with a nil Origin representing “use whatever you have”.

However, that turns out to be fairly bug-prone: if we forget
to populate an Origin somewhere, we end up with an incomplete
check instead of a non-reusable origin (see #61415, #61423).

As of CL 543155, the reusability check for a given query
now depends on what is needed by the query more than what
is populated in the origin. With that in place, we can simplify
the handling of the Origin struct by using a nil pointer
to represent inconsistent or unavailable origin data, and
otherwise always reporting whatever origin information we have
regardless of whether we expect it to be reused.

Updates #61415.
Updates #61423.

Change-Id: I97c51063d6c2afa394a05bf304a80c72c08f82cf
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/543216
Auto-Submit: Bryan Mills <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>
Reviewed-by: Michael Matloob <[email protected]>
@bcmills bcmills mentioned this issue Nov 21, 2023
Closed
@bcmills bcmills mentioned this issue Mar 4, 2024
Open
@golang golang locked and limited conversation to collaborators Nov 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@bcmills bcmills

Labels
FrozenDueToAge GoCommand cmd/go modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Milestone
Go1.22
Development

No branches or pull requests
2 participants
@bcmills @gopherbot