cmd/compile: compiling large functions is slow

[josharian]

Chopping up large functions into smaller ones can have a significant impact on compile
time. See e.g. chan/select5.go (CL 72590045) and the bottom of rotate.go.

[gopherbot]

Comment 1 by zondolfin:

I created a gigantic type switch with around 3000 cases and 40k LOC, and when compiling
it got
$ time go build
real 2m12.840s
user 2m9.947s
sys 0m1.776s
To find this code, check out
https://github.com/zond/godec/tree/97525c53ef42889c5f9f96033030dd4675047b21, run "go
generator/generator.go"
Then I broke up the switch by moving the body of the cases to other functions, so that
each case only called another function instead of allocate, for loop etc, and got
$ time go build
real 0m8.115s
user 0m7.812s
sys 0m0.282s
To find THAT code, check out
https://github.com/zond/godec/tree/a0188c898f1197709a1238fba5b5864f63eac552, run "go run
generator/generator.go"

[josharian]

My original issue ("large functions compile slowly") is too vague and generic to be helpful.

However, the code in Comment 1 is a useful test case. As of 1.8rc1, we spend ~20% of CPU time in resolveFwdRefs and ~10% of CPU time in liveness bit twiddling.

I haven't looked at resolveFwdRefs, but the liveness bit twiddling should be readily reduced. The problem is that we are doing O(number of variables) work for every instruction in the program in livenessprologue. But in almost all cases, progeffects only flips a bit or two. Switching to a sparse return value from progeffects (with appropriate caching and allocation avoidance) should help considerably. I may mail a CL that does this for 1.9.

[josharian]

And looks like improving the liveness bit twiddling helps regular code a little bit as well. My crude prototype shows:

name       old time/op      new time/op      delta
Template        212ms ± 4%       208ms ± 2%  -1.67%        (p=0.000 n=23+25)
Unicode        96.9ms ± 4%      96.5ms ± 4%    ~           (p=0.358 n=23+24)
GoTypes         665ms ± 4%       659ms ± 4%    ~           (p=0.068 n=25+25)
Compiler        2.83s ± 2%       2.79s ± 2%  -1.44%        (p=0.000 n=23+25)

name       old user-ns/op   new user-ns/op   delta
Template   265user-ms ± 3%  262user-ms ± 2%  -1.08%        (p=0.001 n=22+25)
Unicode    135user-ms ± 3%  135user-ms ± 5%    ~           (p=0.512 n=21+25)
GoTypes    889user-ms ± 3%  880user-ms ± 2%  -1.04%        (p=0.006 n=25+24)
Compiler   3.85user-s ± 2%  3.81user-s ± 1%  -1.23%        (p=0.000 n=23+25)

[gopherbot]

CL https://golang.org/cl/36112 mentions this issue.

[gopherbot]

CL https://golang.org/cl/36211 mentions this issue.

[josharian]

The last two commits helped compile time for this code considerably. I don't see anything further to be done peculiar to this code--the rest is now general and known compiler slowness.