-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Use Labeled
blocks in TailRec
, instead of label-defs.
#5115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hello, and thank you for opening this PR! 🎉
All contributors have signed the CLA, thank you! ❤️
Commit Messages
We want to keep history, but for that to actually be useful we have
some rules on how to format our commit messages (relevant xkcd).
Please stick to these guidelines for commit messages:
- Separate subject from body with a blank line
- When fixing an issue, start your commit message with
Fix #<ISSUE-NBR>:
- Limit the subject line to 72 characters
- Capitalize the subject line
- Do not end the subject line with a period
- Use the imperative mood in the subject line ("Add" instead of "Added")
- Wrap the body at 80 characters
- Use the body to explain what and why vs. how
adapted from https://chris.beams.io/posts/git-commit
Have an awesome day! ☀️
Can we avoid the def fact(n: Int, acc: Int): Int = {
var n$tailLocal1: Int = n
var acc$tailLocal1: Int = acc
var result$tailLocal1: Int = _
tailLabel1[Unit]: {
while (true) {
if (n$tailLocal1 == 0) {
result$tailLocal1 = acc$tailLocal1
return[tailLabel1] ()
} else {
val n$tailLocal1$tmp: Int = n$tailLocal1 - 1
val acc$tailLocal1$tmp: Int = n$tailLocal1 * acc$tailLocal1
n$tailLocal1 = n$tailLocal1$tmp
acc$tailLocal1 = acc$tailLocal1$tmp
}
}
}
result$tailLocal1
} EDIT: Introduced a new local variable for the result |
No, in general we cannot do that, because the right-hand-side to the inner return can refer to values that are not in scope at the top-level of the method. Besides, if you have several exit points, it doesn't really work either. It would be possible to do something like this: def fact(n: Int, acc: Int): Int = {
var tailResult: Int = 0
var n$tailLocal1: Int = n
var acc$tailLocal1: Int = acc
tailReturn1[Unit]: {
while (true) {
tailLabel1[Unit]: {
tailResult = {
if (n$tailLocal1 == 0) {
acc
} else {
val n$tailLocal1$tmp: Int = n$tailLocal1 - 1
val acc$tailLocal1$tmp: Int = n$tailLocal1 * acc$tailLocal1
n$tailLocal1 = n$tailLocal1$tmp
acc$tailLocal1 = acc$tailLocal1$tmp
return[tailLabel1] ()
}
}
return[tailReturn1] ()
}
}
}
tailResult
} but that is actually less efficient than leaving the dead code, even if less "ugly". I also thought that instead of generating |
A more reliable alternative would be to allow |
b920577
to
a64ba26
Compare
c2aa05f
to
4adc949
Compare
Labeled
blocks in TailRec
, instead of label-defs.Labeled
blocks in TailRec
, instead of label-defs.
Rebased. Now ready for full review. |
I think there is alternate translation that we may want to consider: def fact(n: Int, acc: Int): Int = {
var n$tailLocal1: Int = n
var acc$tailLocal1: Int = acc
tailReturn1[Int]: {
while (true) {
tailLabel1[Unit]: {
return tailReturn1[Int] {
if (n$tailLocal1 == 0) {
acc
} else {
val n$tailLocal1$tmp: Int = n$tailLocal1 - 1
val acc$tailLocal1$tmp: Int = n$tailLocal1 * acc$tailLocal1
n$tailLocal1 = n$tailLocal1$tmp
acc$tailLocal1 = acc$tailLocal1$tmp
return[tailLabel1] ()
}
}
}
}
} @sjrd Can you clarify why we need temporaries for arguments of tail recursive call? I.e. why val n$tailLocal1$tmp: Int = n$tailLocal1 - 1
val acc$tailLocal1$tmp: Int = n$tailLocal1 * acc$tailLocal1
n$tailLocal1 = n$tailLocal1$tmp
acc$tailLocal1 = acc$tailLocal1$tmp instead of:
|
That does not work. You'll still need a dead code
In your alternative translation without temporaries, the computation of the new value for |
I added a commit with my own alternative proposal to use |
test performance please |
Looks like the performance bot doesn't like me :p |
test performance please |
performance test scheduled: 1 job(s) in queue, 0 running. |
@liufengyun can you add @sjrd to the whitelist for running benchmarks ? |
Performance test finished successfully: Visit http://dotty-bench.epfl.ch/5115/ to see the changes. Benchmarks is based on merging with master (fd93b46) |
Now the bot is for your service @sjrd . |
@liufengyun Thank you :) Since you're there, it seems the performance test launched above had an issue: the page it points to is a 404 for me :s (http://dotty-bench.epfl.ch/5115/) |
It is an issue with Github pages. Sometimes you can use the URL
http://dotty-bench.epfl.ch/5115/index.html , sometimes you need to wait a
little while before it's available.
Le Wed, Oct 3, 2018 à 2:25 PM, Sébastien Doeraene <[email protected]>
a écrit :
… @liufengyun <https://github.com/liufengyun> Thank you :)
Since you're there, it seems the performance test launched above had an
issue: the page it points to is a 404 for me :s (
http://dotty-bench.epfl.ch/5115/)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5115 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAuDyQecAanMXxY1FubJLpuynatKoX_Vks5uhKyygaJpZM4WshD9>
.
|
52d8c77
to
abaf97e
Compare
Rebased. |
What's the status here. Can this be merged? |
It needs a review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should also add the missing case for While
in the tree traversal:
case tree @ While(cond, body) =>
tpd.cpy.While(tree)(noTailTransform(cond), noTailTransform(body))
And a test that exercises it
val varsForRewrittenParamSyms = transformer.varsForRewrittenParamSyms | ||
|
||
val initialValDefs = { | ||
val initialParamValDefs = for ((param, local) <- rewrittenParamSyms.zip(varsForRewrittenParamSyms)) yield { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(rewrittenParamSyms, varsForRewrittenParamSyms).zipped
to avoid intermediate collection?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did not even know about this thing 🤣
val rewrittenParamSyms = transformer.rewrittenParamSyms | ||
val varsForRewrittenParamSyms = transformer.varsForRewrittenParamSyms | ||
|
||
val initialValDefs = { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
initialVarDefs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. Depends on what the coding style in Dotty is. There are var
definitions, but they are still tpd.ValDef
s because there is no such thing as a VarDef
. That's why I called them initialValDefs
. Is it customary to refer to tpd.ValDef
s that represents var
s as VarDef
s?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quickly going through the codebase, I cannot infer a coding style. No strong feeling about this one
|
||
val rhsFullyTransformed = varForRewrittenThis match { | ||
case Some(localThisSym) => | ||
val classSym = tree.symbol.owner.asClass |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is equivalent to owner
defined on line 99
var continueLabel: Option[Symbol] = None | ||
var varForRewrittenThis: Option[Symbol] = None | ||
var rewrittenParamSyms: List[Symbol] = Nil | ||
var varsForRewrittenParamSyms: List[Symbol] = Nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you could simplify the implementation here. If I understand correctly rewrittenParamSyms
is just paramSyms
that you reorder (for reasons I don't understand). Would it be possible to get rid of rewrittenParamSyms
? Something like:
private[this] var myVarsForRewrittenParamSyms: List[Symbol] = _
def varsForRewrittenParamSyms(implicit ctx: Context) = {
if (myVarsForRewrittenParamSyms == null) {
myVarsForRewrittenParamSyms = paramSyms.map(param =>
ctx.newSymbol(method, TailLocalName.fresh(param.name.toTermName), Flags.Synthetic | Flags.Mutable, param.info))
}
myVarsForRewrittenParamSyms
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, rewrittenParamSyms
is a subset of paramSyms
. It only contains parameters whose value is changed in at least one tail-recursive call. This is quite important to avoid declaring and assigning useless variables.
var continueLabel: Option[Symbol] = None | ||
var varForRewrittenThis: Option[Symbol] = None | ||
var rewrittenParamSyms: List[Symbol] = Nil | ||
var varsForRewrittenParamSyms: List[Symbol] = Nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These four variables would benefit from comments explaining what they are
} | ||
|
||
val tpt = TypeTree(method.info.resultType) | ||
Block(assignments, Typed(Return(Literal(Constant(())).withPos(tree.pos), ref(getContinueLabel())), tpt)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
seq(assignments, ...)
* while (true) { | ||
* tailResult[ResultType]: { | ||
* return { | ||
* // original rhs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I understand correctly, it is not original rhs here. It is original rhs with tail recursive call rewritten as you describe below
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is true, but if you follow the "story" in the comment, it later says
Self-recursive calls in tail-position are then replaced by
So at this point in the story, it is still the original rhs.
I can also rewrite the story to make everything happen "at the same time", if you prefer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's very helpful to have a self contained example that describes the whole transformation
} | ||
|
||
val tpt = TypeTree(method.info.resultType) | ||
Block(assignments, Typed(Return(Literal(Constant(())).withPos(tree.pos), ref(getContinueLabel())), tpt)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe it is worth adding a method to tpd
:
def Return(expr: Tree, from: Symbol)(implicit ctx: Context): Return =
Return(expr, Ident(from.termRef))
I see multiple repetitions of Return(..., ref(...))
and Return(..., Ident(... .termRef))
(I believe both are equivalent, but the latter is more efficient)
} | ||
|
||
val tpt = TypeTree(method.info.resultType) | ||
Block(assignments, Typed(Return(Literal(Constant(())).withPos(tree.pos), ref(getContinueLabel())), tpt)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is the Return
tree wrapped in a TypedTree
node?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To perfectly preserve the type of the node (otherwise it's Nothing
), which in turn is very important for the TypeAssigner
not to infer different types as lubs of branches/cases when one of the branches has been replaced by a return
. Otherwise Ycheck failures arise.
|
||
import dotty.tools.dotc.ast.tpd._ | ||
|
||
var rewrote: Boolean = false | ||
|
||
var continueLabel: Option[Symbol] = None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is weird to use an Option
here since the value is not really optional. Here is the convention we tend to follow for idempotent lazily computed values:
private[this] myContinueLabel: Symbol = _
def continueLabel(implicit ctx: Context) = {
if (myContinueLabel == null) {
myContinueLabel = ...
}
myContinueLabel
}
* } | ||
* } | ||
* </pre> | ||
* <p> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While you're at it, you could remove the html tags in this comment, it should use markdown instead.
@@ -486,7 +486,7 @@ trait TypeAssigner { | |||
tree.withType(defn.NothingType) | |||
|
|||
def assignType(tree: untpd.WhileDo)(implicit ctx: Context): WhileDo = | |||
tree.withType(defn.UnitType) | |||
tree.withType(if (tree.cond eq EmptyTree) defn.NothingType else defn.UnitType) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does this occur ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because we want a way to represent an infinite while
loop, whose type is therefore Nothing
. It's the whole point of the second commit in this PR.
abaf97e
to
437a828
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated. AFAICT I should have addressed the points raised in the review.
case _ :: _ => | ||
val (tempValDefs, assigns) = (for ((lhs, rhs) <- assignThisAndParamPairs) yield { | ||
val temp = c.newSymbol(method, TailTempName.fresh(lhs.name.toTermName), Flags.Synthetic, lhs.info) | ||
(ValDef(temp, rhs), Assign(ref(lhs), ref(temp)).withPos(tree.pos)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't work, because it does not use the correct owner
for the new symbol.
8f408bb
to
92458fc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment is not addressed:
We should also add the missing case for
While
in the tree traversal:
case tree @ While(cond, body) =>
tpd.cpy.While(tree)(noTailTransform(cond), noTailTransform(body))
And tests that exercise it
Although this could be addressed as separate PR. Otherwise LGTM!
Let's discuss in the meeting if we want to keep the last commit
val initialVarDefs = { | ||
val initialParamVarDefs = (for ((param, local) <- (rewrittenParamSyms, varsForRewrittenParamSyms).zipped) yield { | ||
ValDef(local.asTerm, ref(param)) | ||
}).toList |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you don't need toList
if you do:
val initialParamVarDefs = (rewrittenParamSyms, varsForRewrittenParamSyms).zipped.map(
(local, param) => ValDef(local.asTerm, ref(param)))
Somehow, for .. yield
is not equivalent to map
here
It's easier to first explain on an example. Consider the following tail-recursive method: def fact(n: Int, acc: Int): Int = if (n == 0) acc else fact(n - 1, n * acc) It is now translated as follows by the `tailrec` transform: def fact(n: Int, acc: Int): Int = { var n$tailLocal1: Int = n var acc$tailLocal1: Int = acc while (true) { tailLabel1[Unit]: { return { if (n$tailLocal1 == 0) { acc } else { val n$tailLocal1$tmp1: Int = n$tailLocal1 - 1 val acc$tailLocal1$tmp1: Int = n$tailLocal1 * acc$tailLocal1 n$tailLocal1 = n$tailLocal1$tmp1 acc$tailLocal1 = acc$tailLocal1$tmp1 (return[tailLabel1] ()): Int } } } } throw null // unreachable code } First, we allocate local `var`s for every parameter, as well as `this` if necessary. When we find a tail-recursive call, we evaluate the arguments into temporaries, then assign them to the `var`s. It is necessary to use temporaries in order not to use the new contents of a param local when computing the new value of another param local. We avoid reassigning param locals if their rhs (i.e., the actual argument to the recursive call) is itself, which does happen quite often in practice. In particular, we thus avoid reassigning the local var for `this` if the prefix is empty. We could further optimize this by avoiding the reassignment if the prefix is non-empty but equivalent to `this`. If only one parameter ends up changing value in any particular tail-recursive call, we can avoid the temporaries and directly assign it. This is also a fairly common situation, especially after discarding useless assignments to the local for `this`. After all that, we `return` from a labeled block, which is right inside an infinite `while` loop. The net result is to loop back to the beginning, implementing the jump. The `return` node is explicitly ascribed with the previous result type, so that lubs upstream are not affected (not doing so can cause Ycheck errors). For control flows that do *not* end up in a tail-recursive call, the result value is given to an explicit `return` out of the enclosing method, which prevents the looping. There is one pretty ugly artifact: after the `while` loop, we must insert a `throw null` for the body to still typecheck as an `Int` (the result type of the `def`). This could be avoided if we dared type a `WhileDo(Literal(Constant(true)), body)` as having type `Nothing` rather than `Unit`. This is probably dangerous, though, as we have no guarantee that further transformations will leave the `true` alone, especially in the presence of compiler plugins. If the `true` gets wrapped in any way, the type of the `WhileDo` will be altered, and chaos will ensue. In the future, we could enhance the codegen to avoid emitting that dead code. This should not be too difficult: * emitting a `WhileDo` whose argument is `true` would set the generated `BType` to `Nothing`. * then, when emitting a `Block`, we would drop any statements and expr following a statement whose generated `BType` was `Nothing`. This commit does not go to such lengths, however. This change removes the last source of label-defs in the compiler. After this commit, we will be able to entirely remove label-defs.
92458fc
to
0c24a34
Compare
Updated to avoid the I have also tried the variant where we cast to I have therefore kept the variant that uses |
Such `WhileDo` node have type `Nothing` instead of `Unit`. This is used in the loops generated by `TailRec` in order not to need some spurious dead code.
0c24a34
to
3084b1a
Compare
override def typedWhileDo(tree: untpd.WhileDo)(implicit ctx: Context): Tree = { | ||
assert((tree.cond ne EmptyTree) || ctx.phase.refChecked, i"invalid empty condition in while at $tree") | ||
super.typedWhileDo(tree) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And here is the addition of an explicit Ycheck
test that WhileDo(EmptyTree, _)
is not used too early, as requested by @smarter.
@@ -452,6 +452,11 @@ class TreeChecker extends Phase with SymTransformer { | |||
tree1 | |||
} | |||
|
|||
override def typedWhileDo(tree: untpd.WhileDo)(implicit ctx: Context): Tree = { | |||
assert((tree.cond ne EmptyTree) || ctx.phase.refChecked, i"invalid empty condition in while at $tree") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why ctx.phase.refChecked
? Why not an equivalent method for TailRec
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because it existed, and the only requirement is that an infinite loop must not be pickled. It doesn't have anything to do with tailrec per se.
Probably fixed by scala#5115
This PR depends on #5113 and #5112.
It's easier to first explain on an example. Consider the following tail-recursive method:
It is now translated as follows by the
tailrec
transform:First, we allocate local
var
s for every parameter, as well asthis
if necessary.When we find a tail-recursive call, we evaluate the arguments into temporaries, then assign them to the
var
s. It is necessary to use temporaries in order not to use the new contents of a param local when computing the new value of another param local.We avoid reassigning param locals if their rhs (i.e., the actual argument to the recursive call) is itself, which does happen quite often in practice. In particular, we thus avoid reassigning the local var for
this
if the prefix is empty. We could further optimize this by avoiding the reassignment if the prefix is non-empty but equivalent tothis
.If only one parameter ends up changing value in any particular tail-recursive call, we can avoid the temporaries and directly assign it. This is also a fairly common situation, especially after discarding useless assignments to the local for
this
.After all that, we
return
from a labeled block, which is right inside an infinitewhile
loop. The net result is to loop back to the beginning, implementing the jump.For control flows that do not end up in a tail-recursive call, the result value is given to an explicit
return
out of the enclosing method, which prevents the looping.There is one pretty ugly artifact: after the
while
loop, we must insert athrow null
for the body to still typecheck as anInt
(the result type of thedef
). This could be avoided if we dared type aWhileDo(Literal(Constant(true)), body)
as having typeNothing
rather thanUnit
. This is probably dangerous, though, as we have no guarantee that further transformations will leave thetrue
alone, especially in the presence of compiler plugins. If thetrue
gets wrapped in any way, the type of theWhileDo
will be altered, and chaos will ensue.In the future, we could enhance the codegen to avoid emitting that dead code. This should not be too difficult:
WhileDo
whose argument istrue
would set the generatedBType
toNothing
.Block
, we would drop any statements and expr following a statement whose generatedBType
wasNothing
.This commit does not go to such lengths, however.
This change removes the last source of label-defs in the compiler. After this commit, we will be able to entirely remove label-defs.