x/telemetry: cmd/go: report maximum support micro architecture by the host of the compiler

[Jorropo]

Proposal Details

More and more architectures are gaining sub-architecture levels under ${GO${GOARCH^^}} variables.

Often higher levels add new instructions and tweak semantics which allows the compiler to emit better code (which would be incompatible with previous cpus).

It would be nice to know what CPUs should be targeted. For example if we discover that 40% of cpus support amd64 v4 considering avx512 routines would be useful.

The main issue with this approach is that the cpu compiling and the cpu running code aren't the same. However I don't think anyone (including me) want to collect telemetry from end binaries.

[thanm]

@pjweinb @hyangah @findleyr per owners

[findleyr]

Thanks for filing this issue, our first telemetry proposal outside the telemetry team (!). In the future, we hope people will fill out the telemetry proposal template, though I know that the process needs documentation, so let's start with what you've provided.

I think one missing piece of information (for me, a naive observer) is the set of unique counters that would be collected. We don't need to decide on the exact format now, but suppose the schema is something like build/goamd64:{v1,v2,v3,v4}. Can you enumerate all of the possible unique values you'd want to collect?

[Jorropo]

That a good question, ideally I would want all the best supported ${GO${GOARCH^^}} values on all archs as more and more architectures now have microarchitecture level:

• all: add GOARM64=v8.1 and so on #60905
• all: add GORISCV64 environment variable #61476
• ...

However a glance at https://telemetry.go.dev/ suggest only amd64 and arm64 are used in any significant number in the wild.

So it is probable that the values for GORISCV64 support would be unique. (bad)

I don't really see value in tracking GOARM64 either as near all the arm64 reports are on darwin and afaik all the apple m{1,2,3} support the same extensions so we wouldn't learn anything right now.

GOAMD64 has good variety, there is a wide variety of generations of products in use today and various products within the same generations can support different extensions (new low power intel chips still only support v2 for example).

---

TL;DR:
build/goamd64:{v1,v2,v3,v4} LGTM.

I don't exactly know the telemetry syntax, but build/goamd64 suggest to me this reads the GOAMD64 variable. I'm not asking for the micro architecture level people are using in the wild today to compile their program, I'm asking for the best level their CPU would support.

I guess most peoples don't even know GOAMD64 exists as it has none to very little impact in the real world so all the samples would be v1 as it's the default.

[the-hotmann]

I guess most peoples don't even know GOAMD64 exists as it has none to very little impact in the real world so all the samples would be v1 as it's the default.

@Jorropo: I think that is not true, but the actual problem is, that golang makes it innessecary hard to use properly.

1. there is no possibility to list the microarchitecture levels (see proposal: all: add GOARCHVERSION instead of "GOARM", "GOARM64", "GOAMD" & "GOAMD64" (etc..) #62129 (comment))
2. there is no possibility to build it properly and clever (hacky bad tricks are needed for such a simple thing - more to this lower *)
3. at runtime it seems hard to inspect: 1. what microarchitecture levels does the hardware support 2. what microarchitecture level was the app compiled for

* About the hacky/bad ways to get the best out of a golang build:

1. my current build process looks like this in the Dockerfile:

ARG	TARGETOS TARGETARCH TARGETVARIANT
RUN	if [ "$TARGETARCH" == "amd64" ] && [ "$TARGETVARIANT" != "" ]; then \
		GOOS=$TARGETOS GOARCH=$TARGETARCH GOAMD64=$TARGETVARIANT CGO_ENABLED=0 go build -ldflags "-s -w" -o ../app -v *.go; \
	elif [ "$TARGETARCH" == "arm" ] && [ "$TARGETVARIANT" != "" ]; then \
		cleanedVariant="${TARGETVARIANT:1}" && \
		GOOS=$TARGETOS GOARCH=$TARGETARCH GOARM=$cleanedVariant CGO_ENABLED=0 go build -ldflags "-s -w" -o ../app -v *.go; \
	else \
		GOOS=$TARGETOS GOARCH=$TARGETARCH CGO_ENABLED=0 go build -ldflags "-s -w" -o ../app -v *.go; \
	fi

Which would be as simple as this, if my proposal would have been accepted:

Version 0: (if TARGETPLATFORM is set, it automatically will be taken as GOPLATFORM)

RUN	CGO_ENABLED=0 go build -ldflags "-s -w" -o ../app -v *.go

Version 1:

RUN	GOPLATFORM=$TARGETPLATFORM CGO_ENABLED=0 go build -ldflags "-s -w" -o ../app -v *.go

Version 2:

ARG	TARGETOS TARGETARCH TARGETVARIANT
RUN	GOOS=$TARGETOS GOARCH=$TARGETARCH GOVARIANT=$TARGETVARIANT CGO_ENABLED=0 go build -ldflags "-s -w" -o ../app -v *.go

GOPLATFORM combines both mandatory (GOOS & GOARCH) and the third optional GOVARIANT option. I have demonstrated here, how exactly this mapping/handling would work:

1. proposal: all: add GOARCHVERSION instead of "GOARM", "GOARM64", "GOAMD" & "GOAMD64" (etc..) #62129 (comment)
2. proposal: all: add GOARCHVERSION instead of "GOARM", "GOARM64", "GOAMD" & "GOAMD64" (etc..) #62129 (comment)
   (it is not "perfect" but as a PoC it should be more than fine)

Also I think that GOAMD64 in general is a super bad idea, aswell as GOARM as both of them do the same (taking a $TARGETVARIANT). This also makes such implementation even more unnessecary complex in the future, when there are more TARGETVARIANTs and more microarchitecture levels on more architectures.

I hope this somewhen changes, but tbh I already have given up. The proposal was closed without a proper explanation and it seems that everyone just implements his own hacky workaround for this. I have already seem quite many different implementations of this, which all solve the very same problem.

This proposal also is "backwards compatible" with the current implementation.

[ianlancetaylor]

@the-hotmann Thanks, but this issue is not the right place for that discussion. This issue is about whether to collect telemetry for GOAMD64 and possibly other CPU-specific environment variables.

[findleyr]

CC @cherrymui -- we revisited this proposal in the telemetry meeting today. Would you find this data to be useful? If so, it seems reasonable to collect.

[cherrymui]

The main issue with this approach is that the cpu compiling and the cpu running code aren't the same. However I don't think anyone (including me) want to collect telemetry from end binaries.

Yeah, I agree that this is the main issue.

I think it may be useful to know what settings people use to build the programs, i.e. what GOAMD64 (etc.) settings they set. E.g. how often GOAMD64=v3 is set.