proposal: cmd/go: add a sub command to manage authentication

[marwan-at-work]

The current state

The Go command utilitizes the ~/.netrc file, both implicitly and explicitly, to authenticate against remote servers when downloading Go code.

Go uses ~/.netrc implicitly when using "direct" downloads because git uses libcurl to fetch dependencies through https which in turn uses the ~/.netrc file to forward credentials.

Go uses ~/.netrc explicitly when authenticating against a GOPROXY server by looking for a matching "machine" URL with a valid login and password and forwards those credentials as a BasicAuth header.

Problem statement

A Go programmer who wants to set their credentials (whether against a proxy or VCS) must know how to create/edit the .netrc file in their home directory manually. This is has a few problems:

1. It is not well documented. The only place I can find a mention of the .netrc file is in the Go FAQ and that is because I was explicitly looking for it.

2. It is not a good UX: you have to learn/follow the netrc syntax to configure your credentials. Furthermore, Go does not validate the .netrc syntax for you. For example, machine myproxy.com login mytoken silently fails and does not send the credentials to myproxy.com unless I explicitly put machine myproxy.com login myuser password mytoken. On the other hand, machine github.com login mytoken works just fine for VCS authentication (since this is handled by git and libcurl directly and not by Go)

3. Most importantly, I found that newcomers to Go find this confusing and hard to deal with in comparison to other languages:

Many languages and tools abstract authentication management in their command line:

1. NodeJS has npm login, npm logout, and npm config set|get which all manage the ~/.npmrc file.

2. Ruby has gem signin and gem signout to manage credentials as well (~/.gem/credentials)

3. Dart's pub command line lets you manage ~/.pub-cache/credentials.json through an interactive browser that signs in to a Google account when running pub publish and it also has pub signout to remove those credentials.

4. Docker (though not a language but certainly has a registry) has docker login and docker logout

5. gcloud (though not a language) can also manage the credentials on the filesystem via gcloud auth login and can be static or interactive

And the list goes on.

Proposal

Go should provide a more pleasant, and less error prone, way to configure user's credentials for downloading private module dependencies.

Specifically, Go should be able to create and edit the ~/.netrc file without the user's direct manipulation of it.

Go should be able to add/edit/remove specific lines in the ~/.netrc through the Go command line.

Examples

Please note: The following syntax is arbitrary and can definitely be changed. This proposal is more about getting agreement that we should let the Go command manage the .netrc file and is not picky about what the syntax will look like.

That said, suggestion on what the command syntax would look like is welcome here.

1. Login to github.com

$ go auth login -host=github.com -user=marwan-at-work -password=myToken 

$ cat ~/.netrc
machine github.com login marwan-at-work password myToken

$ echo "myToken" | go auth login -host=myproxy.com -user=marwan --password-stdin

$ cat ~/.netrc
machine github.com login marwan-at-work password myToken
machine myproxy.com login marwan password myToken

2. Logout of github.com

$ go auth logout github.com

$ cat ~/.netrc
machine myproxy.com login marwan password myToken

3. List current authentications

$ go auth list # or go list auth

cc: @heschik @bcmills @jayconrod (I added the modules label, but I don't think it's exclusively for modules so I'm not sure what other label this might fit into)

Thank you!

[jayconrod]

To address a couple points:

1. Lack of documentation is certainly a problem. Hopefully, the upcoming module reference documentation will address that, specifically in CL 240686. We may want to add other how-to pages on that.
2. About silent failures with .netrc parsing: that seems like a bug we should fix in any case, regardless of what's decided here. Mind filing an issue?

That said, I'm not sure we should rely on .netrc as our primary authentication mechanism, long-term. Until we decide that, we should hold off adding more explicit support on the command line. The main problems are 1) no support for whitespace characters in passwords, 2) no support for other authentication methods (cookies, SSH keys, etc.), 3) no support for module-specific or repository-specific credentials.

#26232 is a feature request for an extensible authentication mechanism in the style of git-credential-helper. @bcmills was working on that last year, but it didn't make it into 1.14, and we didn't get back to it for 1.15.

That would provide a lot of flexibility, but I'm not sure it would make the user experience any simpler. Perhaps a go auth (or go mod auth) command could be built together with that?

[marwan-at-work]

About silent failures with .netrc parsing: that seems like a bug we should fix in any case, regardless of what's decided here. Mind filing an issue?

Done: #40215

That would provide a lot of flexibility, but I'm not sure it would make the user experience any simpler

Agreed. The GOAUTH env var would probably be even more complex for users to configure so a go auth command would hopefully mitigate the complexity here.

Perhaps a go auth (or go mod auth) command could be built together with that?

Could we not have a go auth command that supports both the current .netrc mechanism for the time being and can be adjusted to support GOAUTH once/if it gets implemented? This would provide a better experience for users who are trying to use this to authenticate now. I definitely understand that .netrc has its limitation, but to Go users, this is the only option at the moment.

Thanks!

[bcmills]

Under the original GOAUTH proposal, git users would be able to use git credential fill, which is approximately as complex (or simple) as cat >> ~/.netrc.

But it's true that git credential erase has no equally-simple analogue at the moment.

[bcmills]

That said, I would prefer to keep the go command orthogonal to its credential stores. For editing .netrc files, I would rather see a .netrc-specific tool than a go-specific one, given that .netrc files may in general contain credentials intended for use with not only the go command but also other tools (such as curl and ftp).

[marwan-at-work]

Under the original GOAUTH proposal, git users would be able to use git credential fill, which is approximately as complex (or simple) as cat >> ~/.netrc. But it's true that git credential erase has no equally-simple analogue at the moment.

From a UX perspective, I don't think is a good idea. A nice gain of having a GOPROXY is that we can tell users they don't need to have "git" and other VCS binaries in their build step because a GOPROXY will take care of module fetching for you. If we had to tell them they need to make sure they have "git" just to authenticate against a GOPROXY, it feels like a step in the wrong direction. But definitely correct me if I missed something.

That said, I would prefer to keep the go command orthogonal to its credential stores. For editing .netrc files, I would rather see a .netrc-specific tool than a go-specific one, given that .netrc files may in general contain credentials intended for use with not only the go command but also other tools (such as curl and ftp).

Agreed. But telling Go developers to install a third party tool just so they can make Go understand and pass credentials is not the most ideal. Though it's certainly better than telling Go developers to hand craft a ~/.netrc file.

Other than the above, I see two options:

1. Have the Go command be able to parse and edit the .netrc file. It can already do the former. This way, if the .netrc is also being used for libcurl/ftp, Go would intelligently edit the file keeping those tools happy.

2. Introduce a .gorc file similar to how most languages work (as I showcased in the issue description). This would ensure that .gorc files are not shared with other tools like curl/ftp and the Go command can take full ownership of it. When GOAUTH is implemented, the .gorc file can simply be one of the credential store options which can then sit next to a more flexible/secure way for managing credentials.

Thanks

[Sylvain2703]

I like the idea of a GOAUTH env variable, for at least a case: GCP Cloud Functions.

According to the Cloud Functions documentation for Go, the only way to use a private Go module is to vendor it (which is very constraining).

Having a GOAUTH env var will allow us to fetch modules from private repos in Google Cloud Functions.
Here is an example:
gcloud functions deploy FunctionName --set-env-vars GOAUTH=user:password@host,GOPRIVATE=gitlab.com/compagny/*"

[salewski]

That said, I'm not sure we should rely on .netrc as our primary authentication mechanism, long-term. Until we decide that, we should hold off adding more explicit support on the command line. The main problems are 1) no support for whitespace characters in passwords, 2) no support for other authentication methods (cookies, SSH keys, etc.), 3) no support for module-specific or repository-specific credentials.

Another problem, as it pertains to HTTP Basic Auth, is that (in accordance with RFC 7617) the user-id cannot contain a colon character (:) because that is the separator character between the <user-id> and <password> fields (and there is no escape mechanism to work around it).

[rsc]

To the extent that this issue is about maintaining ~/.netrc, it sounds like we should encourage prototyping a netrc manager outside the Go command.

It would be nice to have more of a problem statement about how people authenticate to their source repos and what they need. There are many things netrc does not address.