proposal: cmd/go: automatically detect GOPATH

[adg]

Setting up a workspace/GOPATH is one of the big hurdles for new Go users. And maintaining (switching between) several workspaces is a common work flow for some Go developers.

This proposal is to make the GOPATH environment variable optional, and to have the go tool detect the GOPATH root on invocation. It would to this by walking up the tree from the cwd to find a directory named src, and setting GOPATH to the parent of that directory.

For new users, the getting started experience is something like:

mkdir -p src/github.com/user/hello
cd src/github.com/user/hello
vim hello.go
go install
hello

or to install something else:

mkdir src
cd src
go get github.com/docker/docker

Thoughts?

[broady]

/cc @jbuberel who was talking about this this morning.

[davecheney]

@adg thanks for raising this proposal. I thought it was going to be more along the lines of "lets have a default GOPATH value (and it might be $HOME)"

mkdir -p src/github.com/user/hello
cd src/github.com/user/hello
vim hello.go
go install
hello

I like this, but I think there are still a number of edge case that have to be explained en rote.

q. why do I need to make a sub directory, can't go just work in my current directory ?
a. no, not really, Go works with packages, and packages are a directory

q. oh, ok, well I put my code in src/hello.go and it didn't work !
a. ahh, well, src is special, you cannot put code directly in src/

q. ok, so I put my code in gocode/hello.go and it didn't work, you told me src/ was special so I didn't use it
a. err, right, sorry, you need to use src/, but you can't put any code in that, try src/hello/hello.go

q. ok, thanks for helping, that worked, but I have a question, why do I need all these directories, shouldn't Go be able to figure this out ?
a. insert long technical explantation about package names being derived from import paths and how src/ would name no import path so technically not be importable into other package, not that that it matters 'cos its a main package

q. what's a package, i just wanted to try helloworld.go ?!?

This is taken pretty much verbatim from my experiences having to justify the $PROJECT/src directory in gb.

[mdempsky]

I'd like to suggest that any automatic strategy to compute GOPATH should work anywhere within $GOPATH, not just within $GOPATH/src. E.g., it should at least also work within $GOPATH/bin and $GOPATH/pkg.

Easy variation of the proposed scheme is instead of looking for a parent directory named "src" (and then using its parent as GOPATH), look for a parent directory with a "src" child directory (and use the parent itself as GOPATH).

Another option would be to add a new identifying file within the root directory. E.g., Bazel uses an empty file named "WORKSPACE" (see "Getting Started with Bazel") and Chromium's GN uses a file named ".gn" with config settings (see "What you need for a minimal GN build").

[minux]

I think we don't need to support $GOPATH/bin and $GOPATH/pkg, because I can't think of a reason why I'd in one of those directory and still use go build/install/test. Just handle the common case looks good to me. I also don't like introducing a new file to mark GOPATH.

[adg]

@davecheney I think at some point people need to learn about packages. I don't think we should design our entire build process around the very first new user experience. Can you suggest a way of avoiding this conversation that doesn't throw out everything we already have? (My gut reaction is "just tell them to put it in "src/hello" in the first place.")

@mdempsky I concur with @minux - I have no idea why you would care about resolving $GOPATH inside pkg or bin. I never change to those directories, ever. I also don't see the purpose in adding special files, when we already have a special directory (src).

Another consideration for this proposal is that it will theoretically break anyone that already has packages whose import paths have a src path component.

[hallas]

👍 This change makes sense. I don't see what a default $GOPATH (for example at $HOME) would solve in terms of making it easier to get started with Go. I think if a user knows about $HOME (its significance) then he is proficient enough to setup a correct $GOPATH.

[davecheney]

@mdempsky I think the idea of "GOPATH is the parent of a directory called src/" works well.

@adg

I think at some point people need to learn about packages.

Agreed.

I don't think we should design our entire build process around the very first new user experience.

I'll have to disagree with you to some degree there 😄

Can you suggest a way of avoiding this conversation that doesn't throw out everything we already have? (My gut reaction is "just tell them to put it in "src/hello" in the first place.")

I think I probably conflated a number of things in my response. From my experience there are several common pain points for newcomers when they see a path like src/github.com/user/hello

1. Why do I have to setup GOPATH, why can't I just put my stuff anywhere I want (or more likely where I put it in $INSERT_PREVIOUS_LANGUAGE). To that, I agree with you, the answers is "Because reasons".
2. The second is why do they have to put things in a subdirectory of a subdirectory src/ ? Again, it's probably better to stick with "reasons".
3. The third is the confusion that putting a github'ish url in the path. We've seen lots of users who think they have to push to github to compile.

Maybe this is just a matter of introducing the terms in order. Something like

---

There are two pieces of information you need to know to write Go code. The first is all Go code must live inside a workspace, we call that GOPATH. A GOPATH workspaces is any directory that contains the following three directories src/, bin/, and pkg/. You can create one like this

mkdir -p gocode/{src,bin,pkg}

So, gocode is your workspace, your GOPATH. gocode/src is where you will put your Go packages (we'll get to that in a second), gocode/bin is where the compiler will place any programs you compile, and gocode/pkg is the location that the compiler will cache files for later reuse to speed up compilation.

As mentioned above, all the Go code you write must be arranged into packages. What is a package ? A package is just a directory. The name of the directory must match the name of the package declaration that every file in that directory shares, that's why we call them packages.

Putting these two pieces of knowledge together, if you want to write a program called hello, it needs to be in the hello package, and knowing that packages need to live inside your workspaces' src/ directory, the path to this package must be gocode/src/hello

---

This explanation obviously trades off some degree of accuracy to avoid getting bogged down in the details, please don't beat me up for this.

[nightlyone]

What about go init [-path] where path defaults to $HOME/gopath and be done with it?

That could initialize the path, create an directory example too with a minimal main + package and fill this with some dummy content working on all major systems. It then also shows what it did (e.g. go get -v -x example/...), where it puts the stuff and what it actually is (e.g. a minimal binary, an example package, etc.)

When the $GOPATH is missing, suggest setting $GOPATH or calling go init to the user, if he doesn't know what we are talking about.

(Linux- )Distributions could then also easily integrate go init, if their policy needs defaults.

(Appropriate defaults for e.g. Windows are left out for the moment but can be easily added)

[mortdeus]

Is there a reason why we can't just build our default gopath in $GOROOT? Just add a $GOROOT/usr/$USER directory which is populated with it's own /bin, /src, /pkg and auto generated by the go tool.

Its one thing to have the developer explicitly state where they specify $GOPATH resides on their system, its another to allow go to automatically dig around for one.

[tylerdmace]

I think the original proposal is fine for those who understand package and workspace structure but I don't believe it to be helpful to new Go users as was suggested. The problem is that it's very likely that new Go users will be creating test directories within existing directory trees that contain code of different languages -- and src is likely to be a directory that exists all over within that same tree.

[mortdeus]

The go tool doesn't care if you have scripts/source written in other programming languages stored inside the same directory it found a set of *.go files in. It only cares that each declared package is confined within its own directory inside "$GOPATH/src".

I "get" what @adg is saying. Ideally when you use the command go install or go build, you have fed the tool a path (either by an explicit declaration or an implicit assumption that $PWD is your intended target). Given that the go tool already knows where the package you want to build resides, all it has to do is hop out into each subsequential parent directory until it finds itself inside a parent directory called "src". Which then the go tool can deduce that ../src is either a gopath that is already known to the go tool, or it's a base directory where the go tool can construct a new gopath if it is found that one doesn't already exist at said location.

I mean it definitely could work. It's just a really hooky programming environment convention if you ask me. It doesn't seem like it would always play nicely with Unix's file system security model either. (you develop your pkg in /usr/src, go installs executable in /usr/bin, which could bring about potential issues like system util name conflicts, or confused package managers, etc etc.

Personally, I think @adg's proposal makes the go tool far too "magic wand"-ey, for the amount of convenience the language would gain from his proposal's implementation.

[tylerdmace]

My concern wasn't so much that the code of other languages would be an issue; only that src is a common directory name and I figured there would be some potential conflicts in just assuming that you were working with a Go package's src directory when in fact it could be something entirely different. You gave a good example, @mortdeus -- /usr/src could be a conflicting directory if they happened to be working out of it. It's not likely but it could happen.