Compiler crashes building Lift

[scabug]

Compiler crashes building Lift with Scala 2.8 20091001

WARNING] Exception in thread "main" java.lang.Error: no-symbol does not have owner
[WARNING] 	at scala.tools.nsc.symtab.Symbols$$NoSymbol$$.owner(Symbols.scala:1870)
[WARNING] 	at scala.tools.nsc.transform.LambdaLift$$LambdaLifter.outer(LambdaLift.scala:75)
[WARNING] 	at scala.tools.nsc.transform.LambdaLift$$LambdaLifter.scala$$tools$$nsc$$transform$$LambdaLift$$LambdaLifter$$$$enclMethOrClass(LambdaLift.scala:106)
[WARNING] 	at scala.tools.nsc.transform.LambdaLift$$LambdaLifter.searchIn$$1(LambdaLift.scala:286)
[WARNING] 	at scala.tools.nsc.transform.LambdaLift$$LambdaLifter.proxy(LambdaLift.scala:299)
[WARNING] 	at scala.tools.nsc.transform.LambdaLift$$LambdaLifter.proxyRef(LambdaLift.scala:315)
[WARNING] 	at scala.tools.nsc.transform.LambdaLift$$LambdaLifter.postTransform(LambdaLift.scala:391)
[WARNING] 	at scala.tools.nsc.transform.LambdaLift$$LambdaLifter.transform(LambdaLift.scala:407)
[WARNING] 	at scala.tools.nsc.ast.Trees$$Transformer$$$$anonfun$$transform$$4.apply(Trees.scala:1434)
[WARNING] 	at scala.tools.nsc.ast.Trees$$Transformer$$$$anonfun$$transform$$4.apply(Trees.scala:1433)
[WARNING] 	at scala.tools.nsc.ast.Trees$$Transformer.atOwner(Trees.scala:1560)
[WARNING] 	at scala.tools.nsc.transform.TypingTransformers$$TypingTransformer.atOwner(TypingTransformers.scala:37)
[WARNING] 	at scala.tools.nsc.transform.TypingTransformers$$TypingTransformer.atOwner(TypingTransformers.scala:30)
[WARNING] 	at scala.tools.nsc.ast.Trees$$Transformer.transform(Trees.scala:1432)
[WARNING] 	at scala.tools.nsc.transform.TypingTransformers$$TypingTransformer.transform(TypingTransformers.scala:52)
[WARNING] 	at scala.tools.nsc.transform.ExplicitOuter$$OuterPathTransformer.transform(ExplicitOuter.scala:230)
[WARNING] 	at scala.tools.nsc.transform.LambdaLift$$LambdaLifter.transform(LambdaLift.scala:407)

Smaller test case (thanks retronym):

object Msg {
  trait T
  
  trait TSeq
  
  object TSeq {
    implicit def fromSeq(s: Seq[T]): TSeq = error("stub")
  }

  def render {
    val msgs: TSeq = (List[(Any, Any)]().flatMap {
      case (a, b) => {
        a match {
          case _ => b match {
            case _ => error("stub")
          }
        }
      }
    } /*: Seq[T] Adding this type annotation avoids the compile error.*/)
  }
}

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-2429?orig=1
Reporter: David Pollak (dpp)
Attachments:

• 0001-test-case-for-2429.patch (created on Oct 21, 2009 10:43:15 PM UTC, 1759 bytes)
• 2429.scala (created on Oct 3, 2009 9:29:08 AM UTC, 885 bytes)
• lift-debug-scalac.patch (created on Oct 3, 2009 1:49:44 PM UTC, 1697 bytes)
• liftweb-nodeseq.patch (created on Oct 3, 2009 9:40:49 AM UTC, 1631 bytes)

[scabug]

@retronym said:
Here's a boiled down version of the error:

object Msg {
  def render {
    val msgs: scala.xml.NodeSeq = List[(Any, Any)]().flatMap {
      case (a, b) => {
        a match {
          case _ => b match {
            case _ => error("stub")
          }
        }
      }
    }
  }
}

[scabug]

@retronym said:
After heating the crucible further:

object Msg {
  trait T
  
  trait TSeq
  
  object TSeq {
    implicit def fromSeq(s: Seq[T]): TSeq = error("stub")
  }

  def render {
    val msgs: TSeq = (List[(Any, Any)]().flatMap {
      case (a, b) => {
        a match {
          case _ => b match {
            case _ => error("stub")
          }
        }
      }
    } /*: Seq[T] Adding this type annotation avoids the compile error.*/)
  }
}

Dave -- the good news is that you can sidestep this bug by adding ': Seq[Node]' to Msg.scala:68. The bad news is that this takes you only as far as this error in the icode phase:

[WARNING] [log icode] Generating class: net.liftweb.builtin.snippet.Menu$$$$anonfun$$item$$1$$$$anonfun$$apply$$18$$$$anonfun$$apply$$19
[WARNING] Exception in thread "main" java.lang.Error: symbol value e does not exist in net.liftweb.builtin.snippet.Menu$$$$anonfun$$item$$1$$$$anonfun$$apply$$18$$$$anonfun$$apply$$19.apply
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:892)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.genLoadQualifier(GenICode.scala:1037)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:797)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:919)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:919)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:919)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:919)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:919)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:465)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:919)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:430)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genStat(GenICode.scala:193)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase$$$$anonfun$$genStat$$1.apply(GenICode.scala:156)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase$$$$anonfun$$genStat$$1.apply(GenICode.scala:155)
[WARNING] 	at scala.collection.LinearSequenceLike$$class.foreach(LinearSequenceLike.scala:84)
[WARNING] 	at scala.collection.immutable.List.foreach(List.scala:29)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.genStat(GenICode.scala:155)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:918)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.scala$$tools$$nsc$$backend$$icode$$GenICode$$ICodePhase$$$$genLoad(GenICode.scala:465)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:124)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase$$$$anonfun$$gen$$1.apply(GenICode.scala:81)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase$$$$anonfun$$gen$$1.apply(GenICode.scala:81)
[WARNING] 	at scala.collection.LinearSequenceLike$$class.foreach(LinearSequenceLike.scala:84)
[WARNING] 	at scala.collection.immutable.List.foreach(List.scala:29)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:81)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:146)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:100)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase$$$$anonfun$$gen$$1.apply(GenICode.scala:81)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase$$$$anonfun$$gen$$1.apply(GenICode.scala:81)
[WARNING] 	at scala.collection.LinearSequenceLike$$class.foreach(LinearSequenceLike.scala:84)
[WARNING] 	at scala.collection.immutable.List.foreach(List.scala:29)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:81)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:91)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase$$$$anonfun$$gen$$1.apply(GenICode.scala:81)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase$$$$anonfun$$gen$$1.apply(GenICode.scala:81)
[WARNING] 	at scala.collection.LinearSequenceLike$$class.foreach(LinearSequenceLike.scala:84)
[WARNING] 	at scala.collection.immutable.List.foreach(List.scala:29)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:81)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:91)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase$$$$anonfun$$gen$$1.apply(GenICode.scala:81)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase$$$$anonfun$$gen$$1.apply(GenICode.scala:81)
[WARNING] 	at scala.collection.LinearSequenceLike$$class.foreach(LinearSequenceLike.scala:84)
[WARNING] 	at scala.collection.immutable.List.foreach(List.scala:29)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:81)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:91)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.gen(GenICode.scala:77)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.apply(GenICode.scala:73)
[WARNING] 	at scala.tools.nsc.Global$$GlobalPhase.applyPhase(Global.scala:329)
[WARNING] 	at scala.tools.nsc.Global$$GlobalPhase$$$$anonfun$$run$$1.apply(Global.scala:307)
[WARNING] 	at scala.tools.nsc.Global$$GlobalPhase$$$$anonfun$$run$$1.apply(Global.scala:307)
[WARNING] 	at scala.collection.Iterator$$class.foreach(Iterator.scala:536)
[WARNING] 	at scala.collection.mutable.ListBuffer$$$$anon$$1.foreach(ListBuffer.scala:282)
[WARNING] 	at scala.tools.nsc.Global$$GlobalPhase.run(Global.scala:307)
[WARNING] 	at scala.tools.nsc.backend.icode.GenICode$$ICodePhase.run(GenICode.scala:66)
[WARNING] 	at scala.tools.nsc.Global$$Run.compileSources(Global.scala:799)
[WARNING] 	at scala.tools.nsc.Global$$Run.compile(Global.scala:882)
[WARNING] 	at scala.tools.nsc.Main$$.process(Main.scala:91)
[WARNING] 	at scala.tools.nsc.Main$$.main(Main.scala:105)
[WARNING] 	at scala.tools.nsc.Main.main(Main.scala)

[scabug]

@retronym said:
Two testcases. Bug2429A hits an error in lambda lift. Bug2429B blows up in icode.

[scabug]

@retronym said:
patch for lift to work around this bug.

[scabug]

David Pollak (dpp) said:
Thanks for the quick diagnosis.

How can I make such a diagnosis in the future so I can hand you little bits of problems rather than huge piles of Lift? I will gladly do work on my side to reduce work on your side.

[scabug]

@retronym said:
patch for lift to open a debug port on scalac, and add scala-compiler to lift classpath for easy IntelliJ debugging.

[scabug]

@retronym said:
The error message was similar to #2316, which I encountered recently porting Scalaz to 2.8. As such, I thought the recent fix for that bug may have triggered this one. As it turns out, they are unrelated.

First step is to enable extra logging from scalac (-Ylog:all). This might tell you which file was being compiled at the time of the error. Also very useful is -Xprint:, which shows you the program tree after a specified compiler phase. This can be too much information before you have a single file test case.

This didn't help in this case, so I set a breakpoint in the compiler where the error was thrown.

The attached patch (lift-debug-scalac.patch) modifies the Lift pom-s to enable remote debugging of scalac, and also to add scala-compiler.jar as a dependency. This makes it very easy debug with IntelliJ. I used the latest Maia EAP r1 and Scala plugin nightly r2. Feel free to use the debugger of your choice at this stage.

- File -> Open Project -> .../liftweb/pom.xml
- Grab a coffee while the indexing completes...
- Maven Projects Tool Window -> 'Download Sources'
- Finish your coffee

You should be able to see sources for the compiler then, assuming they were avaiable on scala-tools maven repo. Verify this with:

- CTRL-N -> 'Symbols'
- Set a breakpoint at line 1870, click in the gutter on the left side...

As a short digression, I shared your pet peeve about not seeing the .scala files in IntelliJ and fixed this a couple months ago. But it is not on by default:

- File -> Settings -> Code Style -> Scala -> Others -> Show Files in Project View. 

Kick of the build:

- Maven Projects Tool Window -> lift -> install  (or use command line.)

When Scalac starts, it should pause with a message like 'Waiting for a connection on port 5005'.

- Run -> Edit Configurations -> '+' -> Remote, -> Name = "Scalac", OK. Choose this configuration, click debug, and wait for the break point.

Work back up the stack trace to find which line of code, method, and file was being processed. Next step is to isolate and minimise the test case. I created stub implementations of everything referenced in the method in question to reproduce the compile error in standalone file, then pruned it back.

-jason

!r1 http://www.jetbrains.net/confluence/display/IDEADEV/Maia+EAP

!r2 http://www.jetbrains.net/confluence/display/SCA/Scala+Plugin+Nightly+Builds

[scabug]

@dragos said:
Thanks for coming up with such a small example. This look similar to #2175 (an undeclared local variable is used in the pattern match translation).

[scabug]

@retronym said:
Note that there are two test cases with distinct errors in https://lampsvn.epfl.ch/trac/scala/attachment/ticket/2429/2429.scala that I extracted from Lift. The second one indeed looks like #2175. The first manifests a bit earlier, but may they may well be brethren.

Of some puzzlement is the way the behavior changes when a type annotation of Seq[T] is added.

Also, In case the resemblance is lost, TSeq in the test cases corresponds to usages of NodeSeq in Lift.

[scabug]

@paulp said:
Replying to [comment:6 dragos]:

Thanks for coming up with such a small example. This look similar to #2175 (an undeclared local variable is used in the pattern match translation).

I don't think 2429B has anything to do with the pattern matcher. The synthetic e$$1 which is crashing it doesn't make its first appearance until lambdalifter, right here:

    def item(text: Bug2429B#TSeq): Bug2429B#TSeq = Bug2429B.this.TSeq.fromSeq(immutable.this.Nil.map({
      (new anonymous class $$anonfun$$item$$1(e$$1): Function1)
    }, immutable.this.List.builderFactory()).$$asInstanceOf[Seq]());

[scabug]

@dragos said:
Replying to [comment:9 extempore]:

Replying to [comment:6 dragos]:

Thanks for coming up with such a small example. This look similar to #2175 (an undeclared local variable is used in the pattern match translation).

I don't think 2429B has anything to do with the pattern matcher. The synthetic e$$1 which is crashing it doesn't make its first appearance until lambdalifter, right here:

It's already there after explicit outer (I renamed 'e' to 'eee' to be easier to spot). There's only one appearance of 'eee', when it's used as a return value from a case branch:

[[syntax trees at end of explicitouter]]// Scala source: 2429.scala
package <empty> {
  final class Bug2429B extends java.lang.Object with ScalaObject {
    def this(): object Bug2429B = {
      Bug2429B.super.this();
      ()
    };
    abstract trait T extends java.lang.Object;
    abstract trait T1 extends java.lang.Object with Bug2429B.T;
    abstract trait TSeq extends java.lang.Object;
    final class TSeq extends java.lang.Object with ScalaObject {
      def this(): object Bug2429B.TSeq = {
        TSeq.super.this();
        ()
      };
      implicit def fromSeq(s: Seq[Bug2429B.T]): Bug2429B.TSeq = scala.this.Predef.error("stub")
    };
    def item(text: Bug2429B.TSeq): Bug2429B.TSeq = Bug2429B.this.TSeq.fromSeq(immutable.this.Nil.map[Bug2429B.T, List[Bug2429B.T]]({
      final <synthetic> class $$anonfun extends java.lang.Object with (Nothing) => Bug2429B.T with ScalaObject {
        def this(): anonymous class $$anonfun = {
          $$anonfun.super.this();
          ()
        };
        final def apply(name: Nothing): Bug2429B.T = {
          var temp1: Bug2429B.T = (null: Bug2429B.T);
          if (temp1.isInstanceOf[Bug2429B.T1]())
            {
              {
                var temp2: Bug2429B.T1 = temp1.asInstanceOf[Bug2429B.T1]();
                {
                  {
                    {
                      eee
                    }
                  }
                }
              }
            }
          else
            {
              {
                {
                  temp1
                }
              }
            }
        }
      };
      (new anonymous class $$anonfun(): (Nothing) => Bug2429B.T)
    }, immutable.this.List.builderFactory[Bug2429B.T]()))
  }
}

{code}
def item(text: Bug2429B#TSeq): Bug2429B#TSeq = Bug2429B.this.TSeq.fromSeq(immutable.this.Nil.map({
(new anonymous class $$anonfun$$item$$1(e$$1): Function1)
}, immutable.this.List.builderFactory()).$$asInstanceOfSeq);
}}

[scabug]

@paulp said:
Replying to [comment:10 dragos]:

It's already there after explicit outer (I renamed 'e' to 'eee' to be easier to spot). There's only one appearance of 'eee', when it's used as a return value from a case branch:

I still can't see the pattern matcher involvement. I assume you're right, but I can't even try to fix it without first figuring out where. When I compile 2429B the context surrounding what you are calling eee is:

                        {
                          val e: Bug2429B.T1 = temp2;
                          body%0(e){
                            e
                          }
                        }

Which looks normal to me, e there is a parameter to a LabelDef.

When I put in the commented out Seq[T] type annotation and diff the output, the first interesting difference isn't until here:

<     def item(text: Bug2429B#TSeq): Bug2429B#TSeq = Bug2429B.this.TSeq.fromSeq(immutable.this.Nil.map({
<       (new anonymous class $$anonfun$$item$$1(e$$1): Function1)
<     }, immutable.this.List.builderFactory()).$$asInstanceOf[Seq]());
---
>     def item(text: Bug2429B#TSeq): Bug2429B#TSeq = Bug2429B.this.TSeq.fromSeq((immutable.this.Nil.map({
>       (new anonymous class $$anonfun$$item$$1(): Function1)
>     }, immutable.this.List.builderFactory()).$$asInstanceOf[Seq](): Seq));

In the crashy case it thinks "class $$anonfun$$item$$1" takes a constructor argument e$$1. But that's not until lambdalifter. If that's somehow leaking from a pattern matcher local variable I could use some clue into how that takes place. In explicitouter it looks like this:

                          val e: Bug2429B.T1 = temp2;
                          body%0(e){
                            e
                          }

In lambdalifter that becomes this:

                        val e: Bug2429B#T1 = temp2;
                        body%0(e){
                          $$anonfun$$item$$1.this.e$$1
                        }

[scabug]

@paulp said:
I narrowed this a bunch further. Here is enough to crash it.

object Oops {  
  implicit def someImplicit(s: Seq[_]): String = error("stub")
  def item: String = Nil map { case e => e  }
}

(You can crash at a different spot by saying "case e: Any => e" instead.)

It still seems to me like it's in lambdalifter, during the free vars traversal. I tried to figure it out and came to the very speculative conclusion that it's some assumption it's making which is true for methods, but not for labels, or vice versa. Or, I could easily believe that the symbols being constructed in the matcher are not set up correctly, but as yet I don't know in what fashion.

[scabug]

@odersky said:
Fixed in r19201