syscall: use Docker container for making auto-generated files

[bradfitz]

The auto-generation of the syscall/z*.go files is a mess.

Their generation depends on the headers installed on the host system, so nobody can reliably generate them without making a bunch of accidental unrelated changes.

This applies equally to pkg syscall and to golang.org/x/sys/unix.

Let's use a Dockerfile in there and then run mkwhatever.{sh,pl,go} file in Docker, and then docker cp the auto-generated files out of the container back to the host.

Recent chaos:

• https://go-review.googlesource.com/#/c/21971/
• https://go-review.googlesource.com/#/c/21582/
• etc (basically any change)

It's less clear what to do about other Unix, like OpenBSD (https://go-review.googlesource.com/#/c/21797/, etc). @mdempsky? Jails? But openbsd is tricky for other reasons: #15227

Windows seems to be fine.

/cc @ianlancetaylor

[mdempsky]

I don't think there's anything fundamental to generating OpenBSD's z*.go files that requires running on an OpenBSD host. It looks like we just need access to the kernel headers, which we could get by either downloading from OpenBSD's cvsweb server or grabbing an appropriate install or source tarball.

The same looks to be true of the other mksysnum_*.pl scripts.

[hirochachacha]

How do you plan to support non-amd64 architectures?

[bradfitz]

@hirochachacha, we're supposed to have builders for all Go architectures, and builders are supposed to be hermetic. So really we could/should just be using the builders to generate these. Dmitry generated the race runtime *.so files using the builders for various architectures. We use the builders to generate releases. Seems doable here too.

[hirochachacha]

@bradfitz Thanks. Are those builders are open to individual developers?
I think this is one of the head aching problem who wants to contribute to x/sys/unix.
For example, current x/sys/unix supports 12 architectures for linux.
If they want to add a new syscall, it's easy. they can use mksyscall.pl directly.
Once, they need to add a constant which is depends on kernel headers,
libc headers, per architecture, the problem is occurred.
It's not a trivial work for individual developers to prepare these environments.
That's the reason I created https://go-review.googlesource.com/#/c/20007/ before.

[hirochachacha]

(Sorry, conclusions are missing)

Although 12 (virtual) machines => 12 cross compilers will reduce some costs, it's still painful.
So, I want a promising alternative.

[bradfitz]

Are those builders are open to individual developers?

gomote access is available to those who ask and we recognize as being regular contributors.

So, I want a promising alternative.

Got a proposal?

[hirochachacha]

gomote access is available to those who ask and we recognize as being regular contributors.

I see.

Got a proposal?

Yeah, I'm convinced, thank you.

[gopherbot]

CL https://golang.org/cl/37943 mentions this issue.

[bradfitz]

/cc @josephlr who did https://golang.org/cl/37943