text/template: builtins global map impacts dead code elimination

[rojer]

What version of Go are you using (go version)?

1.13.5

Does this issue reproduce with the latest release?

yes

The map of builtin functions that can be invoked from templates is defined as a global variable here.
It references the call function, which uses reflect.Call to invoke a user function. This is one of the conditions that disables dead code elimination in the linker.
At the moment Go is unable to elide unused global maps - see #31704 and long discussion on #2559.

As a result, if anything imports text/template, even transitively, dead code elimination is disabled for entire program.
Consider this simple program: https://github.com/rojer/go-dce-bug
pkg1.MyStruct.Junk method is unused and should be eliminated. However, as described here, DCE is disabled and it is not eliminated:

$ go build && go tool nm go-dce-bug | grep -E '(Junk|unused)'
  4c5710 T github.com/rojer/go-dce-bug/pkg1.(*MyStruct).Junk
  524d20 R github.com/rojer/go-dce-bug/pkg1.(*MyStruct).Junk.stkobj

The reason is, pkg2 imports text/template. the function that references text/template is unused, and even if it were, it does nit use templates as such, just a simple exported function. But since compiler cannot properly elide global map initializers, reflect.Call ends up being marked reachable via text/template.builtins and text/template.call.

Bottom line is, the text/template.builtins map has very high cost to the program and until Go compiler gets better at eliminating unused global variables, its initalization should be removed from package scope and the map should be initialized on first use.

[bradfitz]

How does lazy initialization help?

The linker doesn't care that it's lazily initialized. The link only cares that reflect.Value.Call (or Value.Method) is used anywhere, even lazily, IIRC.

As a result, if anything imports text/template, even transitively, dead code elimination is disabled for entire program.

That's not true.

Only dead code elimination of exported methods.

[rojer]

@bradfitz if the map is only initalized when it's first used, DCE will only be disabled if the program uses templating functions. the problem that currently using any function from text/template is enough, and the code that uses it does not even need to be alive by itself - pkg2.unused in the example project is completely unused, yet it causes text/template.init to be retained, and method DCE to be disabled.
if you want real example of this, see my comment on the issue in u-root - u-root/u-root#1477

Only dead code elimination of exported methods.

this is true, but it's still a lot. for u-root, it's 15% of binary size.

[bcmills]

This is arguably a defect in the DCE analysis: if the code to initialize the map only writes to that map (and does not mutate other global state as a side-effect), and no live function reads from the map, then both the map itself and the code to initialize it should be considered dead and eliminated.

However, that defect is admittedly not easy to fix in general: it would require DCE to do something akin to escape analysis and/or pure-function detection, and the calls to reflect.ValueOf within addValueFuncs are likely not free of side-effects.

[gopherbot]

Change https://golang.org/cl/210284 mentions this issue: text/template: avoid a global map to help the linker deadcode eliminate

[rojer]

@bradfitz can you please clarify what release your pending CL will be committed to? i see that it didn't make 1.14, so which will it be?

[bradfitz]

Go 1.15, August 2020.

[rojer]

ugh, that's bad news

[bradfitz]

Just fork the package and fix meanwhile if you need it. That's an easy workaround which is why we don't rush this in.

[rojer]

how do you suggest we do that? i tried the most obvious way - via vendor/text/template, but it won;t build due to use of internal packages:

vendor/text/template/exec.go:10:2: use of internal package internal/fmtsort not allowed