cmd/compile: rewrite rulegen to not generate plaintext Go code directly

[mvdan]

cmd/compile/internal/ssa/gen/rulegen.go is a program that generates the code for all the SSA rewrite rules.

The way it currently works, the code is almost always generated directly. This keeps the code simple, but also limits it quite a bit. For example, it's hard to tell if a variable will be used or not, so declarations like typ := &config.Types tend to be followed by _ = typ.

It also means we tend to write repetitive and verbose code. For example, these two if statements could be collapsed like if X || Y { break }:

if z1.Op != OpAMD64SHRLconst {
        break
}
if z1.AuxInt != 31 {
        break
}

@josharian tried to do this last bit with string handling in https://go-review.googlesource.com/c/go/+/167438, but that turns into spaghetti code that noone can maintain.

There are other ways we could post-process the code to simplify it, too. For example, we could move declarations closer to their uses, and we could remove unused variable declarations instead of using so many _ = foo safeguards.

We could use go/ast for this, but that's probably too complex for our needs. The generated code only uses a tiny subset of Go, and we could have nodes for higher-level concepts like "rewrite this value".

The last step of the program would be to stringify the node tree, and pass it through go/format as usual.

The purpose of this issue is first to gather feedback. If it sounds like a good idea, I'd like to begin work on this soon, hopefully before the freeze is here again. The first incremental step would probably be to rewrite rulegen to use the intermediate node tree, without changing the generated code at all.

/cc @josharian @mundaym @cherrymui @randall77 @dr2chase

[mvdan]

I forgot to mention; we should synchronize so that this doesn't cause painful conflicts with other rulegen CLs. Ideally, I wouldn't start work until all outstanding rulegen CLs are merged, and any further rulegen work would be delayed until the rewrite is finished. I should need about a week to rewrite the program.

[josharian]

As long as the result ends up being reasonably clear and simple (as I believe it can be), this sounds fine to me.

[randall77]

I'm not really clear on what the goal of this is.

1. Make the generated code more readable.
2. Make the generated code compile faster.
3. Make the generated code run faster.

I don't think the rewrites you suggested help much with 3, as the compiler does effectively those rewrites internally anyway.

It would probably help 2, but I don't think the compile time of the compiler is much of an issue.

I don't really see much benefit for 1 either. Not many people are reading the generated code.

Or is there something else?

[josharian]

The motivation for me is that the giant rewrite files make my editor and git client unhappy.

I also actually care a little bit about 1 and 2, both of which I believe this could help with a bit.

[mvdan]

For some quick numbers: we have ~300k generated lines of code for the rules. Josh's CL 167438 removes ~20k, or ~6%. I presume we could realistically shave off another 5-10% elsewhere, by removing more unnecessary lines and simplifying statements.

As another data point, I sent CL 167399 yesterday, which shaves off ~4k, or ~1.5%. Its implementation would be simpler if we had an intermediate step; the way I hack this together with the plaintext is by using two layers of buffers, which isn't great.

[mvdan]

Ideally, I wouldn't start work until all outstanding rulegen CLs are merged, and any further rulegen work would be delayed until the rewrite is finished. I should need about a week to rewrite the program.

Assuming that work on rulegen stalls during the freeze, perhaps I could take the first week of the 1.13 freeze to do this. We can get it merged shortly after, since the generated code should be unaffected - making the change safe even during the freeze. That should minimize the amount of people I'm blocking with the rewrite, I think.

[randall77]

I think @mvdan is actually arguing for 4., that this change would help make further rulegen changes easier. I'm on board with that one; rulegen changes are currently hard.

I'm convinced, let's do it.

[mvdan]

Is anyone planning rulegen changes before the freeze? I only see https://go-review.googlesource.com/c/go/+/167438, which @josharian labeled as "DO NOT SUBMIT" precisely because it's far too complex to do before the rewrite.

Perhaps we can do this before the freeze, if no further changes to rulegen are planned. Though dotGo is in just three days, so the earliest I could start is late next week.

[randall77]

I've been playing with this one https://go-review.googlesource.com/c/go/+/135420 but I don't think I'm going to have time to make it happen for 1.13.