doc: clarify fetching Go 1.4 from git

[tildeleb]

A decision seems to have been made to no longer provide version controlled 1.4.x source code for doing a clean Go bootstrap on macOS. Is this true? You need a special tarball? How should the the current state of affairs for source code bootstrapping be characterized?

I think the comment that kinda shook me was:
@ianlancetaylor
I suppose I am mildly in favor of doing a 1.4.4 release for better Darwin support going forward.

Currently to get close to a clean bootstrap you have to download a special tarball. A doc change was made to fix the bootstrap issue. But on my system that special version of 1.4.3 builds but doesn't pass tests. It will build 10.2 however.

What really gets me in the craw is the atavistic reversion to a tarball instead of using git and version control.

Yes, I know I could have used my existing Go 10.1 to compile 10.2. Is that what everyone else does now?

Perhaps the pain level of maintaining the 1.4 bootstrap has been reached? Maybe it is time to retire the 1.4 bootstrap and just use archived binaries of previous versions of Go? I think there is some risk and complexity with that approach too, but it's a valid approach, and I could live with it. Whatever the decision is, it should be an explicit decision, not something the team slumps into because no one wants to maintain the bootstrap anymore.

tl;dr version below.
I recently got a new MacBook Pro. Previously I was running macOS 10.10 on my beloved MBP 17" but the new laptop came with High Sierra, macOS 10.13.5 and the Go team had just issued 10.2. Time to update. I am a build from source kinda guy so I did the git pull and all.bash.

Building Go cmd/dist using /Volumes/std/usr/leb/go1.4.
failed MSpanList_Insert 0x783000 0xffd30d60dd0 0x0
fatal error: MSpanList_Insert

Check to see if there is a newer bootstrap version. There is a 1.4.3. Get that. Build it. Same thing.

Look around, it's a High Sierra issue and Go issue 16352 seems to have the answers.

#16352
#17335
#17182

So now I need to download a tarball to bootstrap? That's what the doc says. Oh, they fixed it with a tarball and a doc change.

To build a bootstrap toolchain from source, use either the git branch release-branch.go1.4 or
go1.4-bootstrap-20171003.tar.gz, which contains the Go 1.4 source code plus accumulated fixes to keep
the tools running on newer operating systems.

Download the tarball. Have some kind of issue but I update Xcode and all is OK. At least it builds this time but now it fails with a timezone issue.

ok  	text/template/parse	0.015s
--- FAIL: TestParseInLocation-11 (0.00s)
format_test.go:202: ParseInLocation(Feb 01 2013 AST, Baghdad) = 2013-02-01 00:00:00 +0000 AST,     want 2013-02-01 00:00:00 +0300 +03

Check that error out. It's a known issue because the timezone data changed and the test isn't fixed in the tarball.

#19457

I replace the `TestParseInLocation in the tarball with the one from 10.2 and it works.

Wow this was painful and when building 10.2 I still get an error on my machine.

--- FAIL: TestRespectSetgidDir (0.00s)
    build_test.go:225:     
moveOrCopyFile("/var/folders/95/zwjzrdwc8xj7yg001s6slck80000gn/T/SetGroupID592840517/setgid",     
"/var/folders/95/zwjzrdwc8xj7yg001s6slck80000gn/T/pkgfile074436576"): want "cp 
/var/folders/95/zwjzrdwc8xj7yg001s6slck80000gn/T/pkgfile074436576 
/var/folders/95/zwjzrdwc8xj7yg001s6slck80000gn/T/SetGroupID592840517/setgid", got "mv 
/var/folders/95/zwjzrdwc8xj7yg001s6slck80000gn/T/pkgfile074436576 
/var/folders/95/zwjzrdwc8xj7yg001s6slck80000gn/T/SetGroupID592840517/setgid"
FAIL
FAIL    cmd/go/internal/work    0.025s

which is also supposedly fixed but has been failing on my previous laptop even with 10.10. I will try to figure out what is going on with that. Maybe something to do with being a member of multiple groups? IDK. That's a different issue and I get the feeling it may be specific to my setup.

[ALTree]

Download the tarball. Have some kind of issue but I update Xcode and all is OK. At least it builds this time but now it fails with a timezone issue.

I suggest you read the bootstrapping instructions carefully. This is what they say:

To build a bootstrap toolchain from source, use either the git branch release-branch.go1.4 or go1.4-bootstrap-20171003.tar.gz, which contains the Go 1.4 source code plus accumulated fixes to keep the tools running on newer operating systems. After unpacking the Go 1.4 source, cd to the src subdirectory, set CGO_ENABLED=0 in the environment, and run make.bash (or, on Windows, make.bat).

See how it says to run make.bash? But you used all.bash, which executes all the standard library tests, which is 1) useless when bootstrapping 2) bound to fail, since we don't backport tests fixes to 1.4

That said, it seems like your main complaint is that the bootstrapping tarball is just, well, a tarball called go1.4-bootstrap, and not a properly tagged git release, called go1.4.4. Can you explain why is this a problem for you? Why is having to

1. clone a repository
2. checkout go1.4.4
3. build it
4. delete it when you're done bootstrapping

better than having to

1. download an archive
2. unpack it
3. build it
4. delete it when you're done bootstrapping

?

[mvdan]

I think that another reason the bootstrap version is kept as a branch and not a tag is that it needs changes over time to keep building. So it is easiest to keep it a rolling release than to advance into versions like 1.4.12, especially when Go 1.4 hasn't been supported for more than two years now.

[josharian]

@tildeleb I can see that you got pretty frustrated. I think this is probably a documentation bug, though. Perhaps you could take a second look and suggest what doc changes might have helped. It is hard to see documentation issues without fresh eyes.

As to why I believe this to be only a documentation issue:

• Go 1.4.x is in version control. It is tracked using only a branch, rather than a branch and a tag, so there are no labeled dot releases after 1.4.3. But you can still use a version control to get the code you want; you just check out the head of a branch rather than a tag. There's no particular reason I see to also tag 1.4.4+ releases, although I'm open to hearing some.

• Official release downloads come in source and in precompiled binary form. Someone who wants to bootstrap from source, though, won't want precompiled binaries. Thus the source-only download for 1.4.x.

• It is fully expected that the full 1.4 test suite does not pass. That's ok. 1.4 at this point is for compiling the compiler, nothing more. As long as it achieves that goal reliably, it is not worth the effort to backport timezone tests and the like. Note that many people who contribute to Go bootstrap from the 1.4 source many times a day, as do most of the builders, so there is plenty of test coverage.

If you're not convinced that this is a documentation bug, I'd like to hear more--calmly, though, please. And to reiterate, doc fixes can be hard for those who are too close to the problem, so suggestions are welcome. Thanks.

[FiloSottile]

Might be worth short-circuiting out of all.bash in release-branch.go1.4 with an error.

[tildeleb]

First, I apologize if there was an attitude with how I worded the issue. I thought some more about it.

I understand the urge to classify this issue as a documentation issue. I think the best way to characterize this issue is a "mindset", "process", or user experience issue but part of it is a documentation issue.

Here's what irks me:

1. No tarballs.

If the 1.4 changes are maintained at HEAD on a branch somewhere then why not just say that and give directions on how to get it?

2. Allowing the bootstrap version of Go to fail tests, even standard library tests, is a slippery slope.

How do I know the tests that fail are the "right" ones? I got the tarball version of 1.4 to pass "all.bash" by cherry picking the changes to a single test. I do understand that 1.4 is only used to build the compiler and that I was supposed to use "make.bash".

Thanks

A little more explanation. tl;dr

To be honest I think part of the problem is just accumulated history and keeping up with changes to Go and the related doc over a very long period of time. Here's my first Go code:

-rw-r--r--@ 1 leb  admin  1543 Nov 14  2009 test.go  

It's been a long time since I built the bootstrap version of Go. Here is the version of 1.4 I was using:

-rwxr-xr-x  1 leb  admin   9492020 Dec 11  2014 go*

I don't even know if I built it as a bootstrap or already had that version of it hanging around.

I actually did read the doc and managed to set the environment variable CGO_ENABLE=0 but admit I missed the make.bash. I don't think I ever used "make.bash", I always use "all.bash".

[ianlancetaylor]

We are not going to try to keep the tests passing on the 1.4 branch. We've discovered that there are many tests that effectively test the environment, and those tests break over time. This happens most obviously with tests that wind up testing timezones, as in the case you report. Keeping those tests working is pointless busywork. The documentation tells people to run make.bash or make.bat, and that is what we support.

Similarly, I don't think we want to start making a new 1.4.n release each time we need to change something to keep up with changing systems. These changes are often highly system specific. Few people want or need to bootstrap from source; we must support these people, but we don't have to fire up a new release each time we make a system-specific change.

I think it would be fine if the docs were more clear about using git to get the branch, for example by providing the exact instructions. I think it's also fine to provide a tarball as we do, as that will be more convenient for most people. Anybody want to try to improve the docs here?

So while I'm sorry you ran into trouble, I do think that this is an issue to be addressed only be improving the documentation.

[gopherbot]

Change https://golang.org/cl/124077 mentions this issue: [release-branch.go1.4] all.bash: skip tests by default