cmd/go: setting a global "-trimpath" untrims some temporary cgo files

[elyscape]

What did you do?

Compile the Certificate.Verify example with a global trimpath:

$ go build -a -gcflags "all=-trimpath=${PWD}" .

The example code, for convenience

package main

import (
	"crypto/x509"
	"encoding/pem"
)

func main() {
	// Verifying with a custom list of root certificates.

	const rootPEM = `
-----BEGIN CERTIFICATE-----
MIIEBDCCAuygAwIBAgIDAjppMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT
MRYwFAYDVQQKEw1HZW9UcnVzdCBJbmMuMRswGQYDVQQDExJHZW9UcnVzdCBHbG9i
YWwgQ0EwHhcNMTMwNDA1MTUxNTU1WhcNMTUwNDA0MTUxNTU1WjBJMQswCQYDVQQG
EwJVUzETMBEGA1UEChMKR29vZ2xlIEluYzElMCMGA1UEAxMcR29vZ2xlIEludGVy
bmV0IEF1dGhvcml0eSBHMjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AJwqBHdc2FCROgajguDYUEi8iT/xGXAaiEZ+4I/F8YnOIe5a/mENtzJEiaB0C1NP
VaTOgmKV7utZX8bhBYASxF6UP7xbSDj0U/ck5vuR6RXEz/RTDfRK/J9U3n2+oGtv
h8DQUB8oMANA2ghzUWx//zo8pzcGjr1LEQTrfSTe5vn8MXH7lNVg8y5Kr0LSy+rE
ahqyzFPdFUuLH8gZYR/Nnag+YyuENWllhMgZxUYi+FOVvuOAShDGKuy6lyARxzmZ
EASg8GF6lSWMTlJ14rbtCMoU/M4iarNOz0YDl5cDfsCx3nuvRTPPuj5xt970JSXC
DTWJnZ37DhF5iR43xa+OcmkCAwEAAaOB+zCB+DAfBgNVHSMEGDAWgBTAephojYn7
qwVkDBF9qn1luMrMTjAdBgNVHQ4EFgQUSt0GFhu89mi1dvWBtrtiGrpagS8wEgYD
VR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAQYwOgYDVR0fBDMwMTAvoC2g
K4YpaHR0cDovL2NybC5nZW90cnVzdC5jb20vY3Jscy9ndGdsb2JhbC5jcmwwPQYI
KwYBBQUHAQEEMTAvMC0GCCsGAQUFBzABhiFodHRwOi8vZ3RnbG9iYWwtb2NzcC5n
ZW90cnVzdC5jb20wFwYDVR0gBBAwDjAMBgorBgEEAdZ5AgUBMA0GCSqGSIb3DQEB
BQUAA4IBAQA21waAESetKhSbOHezI6B1WLuxfoNCunLaHtiONgaX4PCVOzf9G0JY
/iLIa704XtE7JW4S615ndkZAkNoUyHgN7ZVm2o6Gb4ChulYylYbc3GrKBIxbf/a/
zG+FA1jDaFETzf3I93k9mTXwVqO94FntT0QJo544evZG0R0SnU++0ED8Vf4GXjza
HFa9llF7b1cq26KqltyMdMKVvvBulRP/F/A8rLIQjcxz++iPAsbw+zOzlTvjwsto
WHPbqCRiOwY1nQ2pM714A5AuTHhdUDqB1O6gyHA43LL5Z/qHQF1hwFGPa4NrzQU6
yuGnBXj8ytqU0CwIPX4WecigUCAkVDNx
-----END CERTIFICATE-----`

	const certPEM = `
-----BEGIN CERTIFICATE-----
MIIDujCCAqKgAwIBAgIIE31FZVaPXTUwDQYJKoZIhvcNAQEFBQAwSTELMAkGA1UE
BhMCVVMxEzARBgNVBAoTCkdvb2dsZSBJbmMxJTAjBgNVBAMTHEdvb2dsZSBJbnRl
cm5ldCBBdXRob3JpdHkgRzIwHhcNMTQwMTI5MTMyNzQzWhcNMTQwNTI5MDAwMDAw
WjBpMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwN
TW91bnRhaW4gVmlldzETMBEGA1UECgwKR29vZ2xlIEluYzEYMBYGA1UEAwwPbWFp
bC5nb29nbGUuY29tMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEfRrObuSW5T7q
5CnSEqefEmtH4CCv6+5EckuriNr1CjfVvqzwfAhopXkLrq45EQm8vkmf7W96XJhC
7ZM0dYi1/qOCAU8wggFLMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAa
BgNVHREEEzARgg9tYWlsLmdvb2dsZS5jb20wCwYDVR0PBAQDAgeAMGgGCCsGAQUF
BwEBBFwwWjArBggrBgEFBQcwAoYfaHR0cDovL3BraS5nb29nbGUuY29tL0dJQUcy
LmNydDArBggrBgEFBQcwAYYfaHR0cDovL2NsaWVudHMxLmdvb2dsZS5jb20vb2Nz
cDAdBgNVHQ4EFgQUiJxtimAuTfwb+aUtBn5UYKreKvMwDAYDVR0TAQH/BAIwADAf
BgNVHSMEGDAWgBRK3QYWG7z2aLV29YG2u2IaulqBLzAXBgNVHSAEEDAOMAwGCisG
AQQB1nkCBQEwMAYDVR0fBCkwJzAloCOgIYYfaHR0cDovL3BraS5nb29nbGUuY29t
L0dJQUcyLmNybDANBgkqhkiG9w0BAQUFAAOCAQEAH6RYHxHdcGpMpFE3oxDoFnP+
gtuBCHan2yE2GRbJ2Cw8Lw0MmuKqHlf9RSeYfd3BXeKkj1qO6TVKwCh+0HdZk283
TZZyzmEOyclm3UGFYe82P/iDFt+CeQ3NpmBg+GoaVCuWAARJN/KfglbLyyYygcQq
0SgeDh8dRKUiaW3HQSoYvTvdTuqzwK4CXsr3b5/dAOY8uMuG/IAR3FgwTbZ1dtoW
RvOTa8hYiU6A475WuZKyEHcwnGYe57u2I2KbMgcKjPniocj4QzgYsVAVKW3IwaOh
yE+vPxsiUkvQHdO2fojCkY8jg70jxM+gu59tPDNbw3Uh/2Ij310FgTHsnGQMyA==
-----END CERTIFICATE-----`

	// First, create the set of root certificates. For this example we only
	// have one. It's also possible to omit this in order to use the
	// default root set of the current operating system.
	roots := x509.NewCertPool()
	ok := roots.AppendCertsFromPEM([]byte(rootPEM))
	if !ok {
		panic("failed to parse root certificate")
	}

	block, _ := pem.Decode([]byte(certPEM))
	if block == nil {
		panic("failed to parse certificate PEM")
	}
	cert, err := x509.ParseCertificate(block.Bytes)
	if err != nil {
		panic("failed to parse certificate: " + err.Error())
	}

	opts := x509.VerifyOptions{
		DNSName: "mail.google.com",
		Roots:   roots,
	}

	if _, err := cert.Verify(opts); err != nil {
		panic("failed to verify certificate: " + err.Error())
	}
}

Then look at the file paths embedded in the file:

$ strings certificate-verify | grep '\.go$' | head -n 5
main.go
/usr/local/Cellar/go/1.10.1/libexec/src/crypto/x509/root_darwin.go
/usr/local/Cellar/go/1.10.1/libexec/src/crypto/x509/pem_decrypt.go
/usr/local/Cellar/go/1.10.1/libexec/src/crypto/x509/root_cgo_darwin.go
/var/folders/yd/l_dq42hj4qn9szd1k_v3hyzw0000gn/T/go-build430556504/b002/_cgo_gotypes.go

What did you expect to see?

For comparison, here's what happens with a local trimpath and without a trimpath:

$ go build -a -gcflags "-trimpath=${PWD}" .
$ strings certificate-verify | grep '\.go$' | head -n 5
main.go
/usr/local/Cellar/go/1.10.1/libexec/src/crypto/x509/root_darwin.go
/usr/local/Cellar/go/1.10.1/libexec/src/crypto/x509/pem_decrypt.go
/usr/local/Cellar/go/1.10.1/libexec/src/crypto/x509/root_cgo_darwin.go
_cgo_gotypes.go
$ go build -a .
$ strings certificate-verify | grep '\.go$' | head -n 5
/Users/eliyoung/private/certificate-verify/main.go
/usr/local/Cellar/go/1.10.1/libexec/src/crypto/x509/root_darwin.go
/usr/local/Cellar/go/1.10.1/libexec/src/crypto/x509/pem_decrypt.go
/usr/local/Cellar/go/1.10.1/libexec/src/crypto/x509/root_cgo_darwin.go
_cgo_gotypes.go

In these cases, _cgo_gotypes.go is listed without an absolute path (or, for that matter, any path).

What did you see instead?

_cgo_gotypes.go is listed with an absolute path to a temporary location.

Additional information

When running the builds with -x, I noticed the following commands in the output (pretty-printed here for readability) when using a global trimpath:

/usr/local/Cellar/go/1.10.1/libexec/pkg/tool/darwin_amd64/compile \
	-o $WORK/b062/_pkg_.a \
	-trimpath $WORK/b062 \
	-trimpath=/Users/eliyoung/private/certificate-verify \
	-p runtime/cgo \
	-std \
	-buildid wTmqLJ-m_g1XXZ2aS_Rm/wTmqLJ-m_g1XXZ2aS_Rm \
	-goversion go1.10.1 \
	-D "" \
	-importcfg $WORK/b062/importcfg \
	-pack \
	-asmhdr $WORK/b062/go_asm.h \
		./callbacks.go \
		./iscgo.go \
		./setenv.go \
		$WORK/b062/_cgo_gotypes.go \
		$WORK/b062/cgo.cgo1.go \
		$WORK/b062/_cgo_import.go

/usr/local/Cellar/go/1.10.1/libexec/pkg/tool/darwin_amd64/compile \
	-o $WORK/b065/_pkg_.a \
	-trimpath $WORK/b065 \
	-trimpath=/Users/eliyoung/private/certificate-verify \
	-p os/user \
	-std \
	-buildid 7nvtUqXU6UAVZC2BVrDl/7nvtUqXU6UAVZC2BVrDl \
	-goversion go1.10.1 \
	-D "" \
	-importcfg $WORK/b065/importcfg \
	-pack \
		./lookup.go \
		./user.go \
		$WORK/b065/_cgo_gotypes.go \
		$WORK/b065/cgo_lookup_unix.cgo1.go \
		$WORK/b065/getgrouplist_darwin.cgo1.go \
		$WORK/b065/listgroups_unix.cgo1.go \
		$WORK/b065/_cgo_import.go

It appears that, when multiple -trimpath flags are passed to go tool compile, the last one wins, but only for cgo-generated files whose names start with an underscore (the other cgo-generated files don't appear to have their names recorded in the binary anywhere).

System details

go version go1.10.1 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/eliyoung/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/eliyoung/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.10.1/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.10.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/yd/l_dq42hj4qn9szd1k_v3hyzw0000gn/T/go-build988645600=/tmp/go-build -gno-record-gcc-switches -fno-common"
GOROOT/bin/go version: go version go1.10.1 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.10.1
uname -v: Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.13.4
BuildVersion:	17E199
lldb --version: lldb-902.0.79.2
  Swift-4.1

[bcmills]

The fix may have the same general structure as the fix for #27681.

Possibly related to #16860.

[bcmills]

It appears that, when multiple -trimpath flags are passed to go tool compile, the last one wins

Indeed; from what I can tell the trimpath flag is defined as an ordinary string flag, not a list:

go/src/cmd/compile/internal/gc/main.go

Line 235 in 8fa789b

flag.StringVar(&pathPrefix, "trimpath", "", "remove `prefix` from recorded source file paths")

[crgimenes]

Getting the -trimpath to receive a list would be great.

[rsc]

Direct use of -gcflags=-trimpath is unsupported.
Closing as a duplicate of #16860.