cmd/go: can't parse go.mod with branch or commit hash specifier

[dwschulze]

What version of Go are you using (go version)?

$ go version
1.15

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOHOSTARCH="amd64"
GOHOSTOS="linux"
Ubuntu 20.04

go env Output

$ go env

What did you do?

Note that I'm using a private server for my git repo. It was created with git init --bare. There is no web server, no github, no gitlab. I have ssh access with my public key and have modified my ~/.gitconfig appropriately. I can access modules as long as I specify a SemVer number, but if I want code from a branch other than master then go doesn't know how to parse the specified for the branch name or the commit hash.

I want to use a branch of a module In go.mod if I specify the branch name I get an error:

require (
	192.168.0.12/gitrepo/go-module-test-dateutil.git @dev2
...
)

 go run main.go
go: errors parsing go.mod:
/home/dean/src/golang/examples/package-module-test/go-module-package-test-drivers/module-driver/go.mod:4:2: require 192.168.0.12/gitrepo/go-module-test-dateutil.git: version "@dev2" invalid: unknown revision @dev2

If I take out the space before the @ I get

require (
	192.168.0.12/gitrepo/go-module-test-dateutil.git@dev2
...
)

$ go run main.go
go: errors parsing go.mod:
/home/dean/src/golang/examples/package-module-test/go-module-package-test-drivers/module-driver/go.mod:4:2: usage: require module/path v1.2.3

I get the same errors if I use the commit hash of the last commit on the branch instead of the branch name.

What did you expect to see?

Should recognize that I'm specifying a branch name or commit hash

What did you see instead?

Parse errors on go.mod

If this is not the correct way to specify a branch or commit what is the correct syntax?

[mvdan]

You're meant to use @branchname with go get, not editing your go.mod directly. Also, see https://golang.org/wiki/Questions.

[dwschulze]

If we are not supposed to manually edit go.mod that needs to be clear in the documentation. Your link says nothing about modules.

If I try go get it also fails:

go get 192.168.0.12/gitrepo/go-module-test-dateutil@dev2
go get 192.168.0.12/gitrepo/go-module-test-dateutil@dev2: unrecognized import path "192.168.0.12/gitrepo/go-module-test-dateutil": https fetch: Get "https://192.168.0.12/gitrepo/go-module-test-dateutil?go-get=1": dial tcp 192.168.0.12:443: connect: connection refused

I have this in my ~/.gitconfig

[url "dean@192.168.0.12:"]
insteadOf = https://192.168.0.12/
[url "git@github.com:"]
insteadOf = https://github.com/

I can use modules from my private repo as long as I do not include a branch specifier with @branchname. Adding @branchname to an entry in go.mo or with go get always fails.

[bcmills]

@dwschulze

I can use modules from my private repo as long as I do not include a branch specifier with @branchname.

What GOPRIVATE and GOINSECURE settings, if any, are you using?

dial tcp 192.168.0.12:443: connect: connection refused

That seems to indicate that the server on 192.168.0.12 either is not serving via HTTPS or is not serving go-import metadata. Can you provide more information about that server?

[dwschulze]

GOPRIVATE and GOINSECURE are both unset. I've tried with GOPRIVATE="192.168.0.12/gitrepo/*" and it makes no difference. GOPRIVATE seems to have no effect at all. I can use modules from my private repos on github with GOPRIVATE unset.

There is no web server on my server at 192.168.0.12. I'm accessing it via ssh. In my ~/.gitconfig

[url "dean@192.168.0.12:"]
insteadOf = https://192.168.0.12/
[url "git@github.com:"]
insteadOf = https://github.com/

The issue arises when using a server with a non-routeable IP address via ssh. As long as I don't specify a branch name it also works.

[bcmills]

If I try go get it also fails:

go get 192.168.0.12/gitrepo/go-module-test-dateutil@dev2

Note that that command is missing the .git suffix on the module path. That's why the go get command is attempting an HTTPS fetch instead of going directly to git.

GOPRIVATE and GOINSECURE are both unset.

In that case, I'm not quite sure how it would be working in the first place. 😅
(Perhaps you're setting GONOSUMDB instead of GOPRIVATE? That could potentially work, but GOPRIVATE seems like a better semantic fit.)

[dwschulze]

That was it

go get 192.168.0.12/gitrepo/go-module-test-dateutil.git@dev2

It made this change to the go.mod:

192.168.0.12/gitrepo/go-module-test-dateutil.git v0.0.3 // indirect

The tag v0.0.3 was on the last commit on the dev2 branch so one problem solved. But if I make another commit on the dev2 branch and run the same go get command to get that latest commit on dev2 it errors (GOPRIVATE is unset at this point):

go get 192.168.0.12/gitrepo/go-module-test-dateutil.git@dev2
go: 192.168.0.12/gitrepo/go-module-test-dateutil.git dev2 => v0.0.4-0.20201106165936-0753fc1f2b10
go get: 192.168.0.12/gitrepo/go-module-test-dateutil.git@v0.0.4-0.20201106165936-0753fc1f2b10/go.mod: verifying module: 192.168.0.12/gitrepo/go-module-test-dateutil.git@v0.0.4-0.20201106165936-0753fc1f2b10/go.mod: reading https://sum.golang.org/lookup/192.168.0.12/gitrepo/go-module-test-dateutil.git@v0.0.4-0.20201106165936-0753fc1f2b10: 410 Gone
server response:
not found: 192.168.0.12/gitrepo/go-module-test-dateutil.git@v0.0.4-0.20201106165936-0753fc1f2b10: invalid version: git fetch -f origin refs/heads/:refs/heads/ refs/tags/:refs/tags/ in /tmp/gopath/pkg/mod/cache/vcs/33d86599dffb0c591e2b31333338860d2c955ee3d237a6391b8891d5644573a4: exit status 128:
fatal: unable to connect to 192.168.0.12:
192.168.0.12[0: 192.168.0.12]: errno=Connection refused

If I set GOPRIVATE it works:

go env -w GOPRIVATE=192.168.0.12/gitrepo/*
$ go get 192.168.0.12/gitrepo/go-module-test-dateutil.git@dev2
go: 192.168.0.12/gitrepo/go-module-test-dateutil.git dev2 => v0.0.4-0.20201106165936-0753fc1f2b10
go: downloading 192.168.0.12/gitrepo/go-module-test-dateutil.git v0.0.4-0.20201106165936-0753fc1f2b10

It made this change to go.mod:

192.168.0.12/gitrepo/go-module-test-dateutil.git v0.0.4-0.20201106165936-0753fc1f2b10 // indirect

So GOPRIVATE has to be set to pull a non-tagged last commit off of a branch, but apparently not if the last commit on the branch is tagged. The docs for GOPRIVATE aren't clear on what it does and when it does something.

The @branchname should only be used with go get, not in a go.mod. The go.mod file has to contain a SemVer based on a tag or a pseudo-tag created by go get @branchname for the last commit on a branch.

Thanks for helping me get this straightened out. The docs need to be clear about this and they are not.

Can you change this to a documentation ticket to get this in the docs?

[bcmills]

The docs need to be clear about this and they are not.

Agreed. That's (part of) #41427.

[bcmills]

So GOPRIVATE has to be set to pull a non-tagged last commit off of a branch, but apparently not if the last commit on the branch is tagged.

That's not how it's supposed to work. Here's a theory, though: perhaps you already had a checksum in your go.sum file (somehow) for v0.0.3, so go get bypassed checksum verification because it already had a checksum?

(That could be the case if, for example, you already depended on v0.0.3 from some go release before Go 1.13, which is when the checksum database was added.)

[bcmills]

The @branchname should only be used with go get, not in a go.mod. The go.mod file has to contain a SemVer based on a tag or a pseudo-tag created by go get @branchname for the last commit on a branch.

The go.mod file can contain a branch name, but it will get rewritten to a pseudo-version. Either way, though, we strongly recommend using go get instead.

The branch behavior is currently documented in go help get and https://golang.org/ref/mod#vcs-branch.

Use of go get (as opposed to editing go.mod files directly) is described in various places, among them https://blog.golang.org/using-go-modules.

Use of GOPRIVATE for non-public modules is documented in go help module-private and https://golang.org/ref/mod#private-modules.

The git suffix is documented in go help importpath and mentioned briefly in https://golang.org/ref/mod#private-module-proxy-direct. We probably ought to duplicate the table of suffixes somewhere in that reference (CC @jayconrod).

---

We still need proper tutorials for a lot of these things (in https://golang.org/doc/tutorial/); @stevetraut is actively working on those.

[bcmills]

Duplicate of #41427