Topic/virt sourcecontext #1

phaller · 2012-01-27T15:06:26Z

Includes a few refactorings in Implicits.scala that we have discussed with Martin. The main change is moving all methods for generating SourceLocations and SourceContexts into a separate trait SourceContextUtils.

… into SourceContextUtils

refactoring sourcecontext support into its own trait

AvlTree performance improvements

This change resolves some issues with ParCtrie splitters and their `remaining` method, which currently evaluates the size of the Ctrie. Since this is still not done lazily, nor in parallel, it has a certain cost, which is unacceptable. Change #1: The `shouldSplitFurther` method is by default implemented by calling the `remaining` method. This method now forwards the call to the same method in the splitter which is by default implemented in the same way as before, but can be overridden by custom collections such as the ParCtrie. Change #2: ParCtrie splitter now has a `level` member which just counts how many times the method has been split. This information is used to override the default `shouldSplitFurther` implementation. Change #3: The tasks and splitters rely heavily on the `remaining` method in the splitter for most operations. There is an additional method called `isRemainingCheap` which returns true by default, but can be overridden by custom collections such as the `Ctrie`.

- relaxed the restrictions on nodes - nodes can be classes, traits and objects, both stand-alone and companion objects -- all are added to the diagram, but usually companion objects are filtered out as they don't have any superclasses - changed the rules for default diagram creation: - classes and traits (and AnyRef) get inheritance diagrams - packages and objects get content diagrams (can be overridden by @contentDiagram [hideDiagram] and @inheritanceDiagram [hideDiagram]) - tweaked the model to register subclasses of Any - hardcoded the scala package diagram to show all relations - enabled @contentDiagram showInheritedNodes by default and changed the setting to hideInheritedNodes (and added a test for this) - better node selection (can select nodes that don't have a corresponding trait) - fixed the docsite link in member selection, which was broken since the first commit :))

Keeps fingerprints in scopes which are bitsets telling you what the last 6 bits of each hashcode of the names stored in the scope are. findMember will avoid looking in a scope if inferprints do not match.

@throws

This change allows an additional notation of the @throws annotation: Old-style: @throws(classOf[Exception]) New-style: @throws[Exception] The optional String argument moves @throws in line with @deprecated, @migration, etc. and prevents confusion caused by the default inheritance of ScalaDoc comments and the non-inheritance of annotations. Before: /** This method does ... * @throws IllegalArgumentException if `a` is less than 0. */ @throws(classOf[IllegalArgumentException]) def foo(a: Int) = ... Now: /** This method does ... */ @throws[IllegalArgumentException]("if `a` is less than 0") def foo(a: Int) = ... ScalaDoc @throws tags remain supported for cases where documentation of thrown exceptions is needed, but are not supposed to be added to the exception attribute of the class file. In this commit the necessary compiler support is added. The code to extract exceptions from annotations is now shared instead of being duplicated all over the place. The change is completely source and binary compatible, except that the code is now enforcing that the type thrown is a subtype of Throwable as mandated by the JVM spec instead of allowing something like @throws(classOf[String]). Not in this commit: - ScalaDoc support to add the String argument to ScalaDoc's exception list - Adaption of the library

Additional test case for Lukas' fix to annotated originals.

We actually need to call normalize here, otherwise we don't progress through #1 below. [infer implicit] scala.this.Predef.implicitly[Higher[Foo.Bar]] with pt=Higher[Foo.Bar] in object Foo 1. tp=Foo.Bar tp.normalize=[A <: <?>]Foo.Bar[A] tp.dealias=Foo.Bar 2. tp=Foo.Bar[A] tp.normalize=Box[A] tp.dealias=Box[A]

Test case for SI-7767

When an application of a blackbox macro expands into a tree `x`, the expansion is wrapped into a type ascription `(x: T)`, where `T` is the declared return type of the blackbox macro with type arguments and path dependencies applied in consistency with the particular macro application being expanded. This invalidates blackbox macros as an implementation vehicle of type providers.

While fixing the problem with the order of typechecks for whitebox expansions, I realized that we’re doing redundant work when expanding blackbox macros. Concretely, typechecking blackbox expansions looked as follows: val expanded1 = atPos(enclosingMacroPosition.focus)(Typed(expanded0, TypeTree(innerPt))) val expanded2 = typecheck("blackbox typecheck #1", expanded1, innerPt) typecheck("blackbox typecheck #2", expanded1, outerPt) Or, if we reformulate it using quasiquotes (temporarily not taking positions into account, since they aren’t important here): val expanded2 = typed(q”$expanded: $innerPt”, innerPt) typed(expanded2, outerPt) In this formulation, it becomes apparent that the first typecheck is redundant. If something is ascribed with some type, then typechecking the ascription against that type does nothing useful. This is also highlights one of the reasons why it would be really nice to have quasiquotes used in the compiler. With them, it’s easy to notice things that would otherwise remain buried behind swaths of boilerplate.

Swathes of important logic are duplicated between `findMember` and `findMembers` after they separated on grounds of irreconcilable differences about how fast they should run: d905558 Variation #10 to optimze findMember fcb0c01 Attempt #9 to opimize findMember. 71d2ceb Attempt #8 to opimize findMember. 77e5692 Attempty #7 to optimize findMember 275115e Fixing problem that caused fingerprints to fail in e94252e Attemmpt #6 to optimize findMember 73e61b8 Attempt #5 to optimize findMember. 04f0b65 Attempt #4 to optimize findMember 0e3c70f Attempt #3 to optimize findMember 41f4497 Attempt #2 to optimize findMember 1a73aa0 Attempt #1 to optimize findMember This didn't actually bear fruit, and the intervening years have seen the implementations drift. Now is the time to reunite them under the banner of `FindMemberBase`. Each has a separate subclass to customise the behaviour. This is primarily used by `findMember` to cache member types and to assemble the resulting list of symbols in an low-allocation manner. While there I have introduced some polymorphic calls, the call sites are only bi-morphic, and our typical pattern of compilation involves far more `findMember` calls, so I expect that JIT will keep the virtual call cost to an absolute minimum. Test results have been updated now that `findMembers` correctly excludes constructors and doesn't inherit privates. Coming up next: we can actually fix SI-7475!

@paulp

[Parts of this patch and some of the commentary are from @paulp] This took me so long to figure out I can't even tell you. Partly because there were two different bugs, one which only arose for trait forwarders and one for mirror class forwarders, and every time I'd make one set of tests work another set would start failing. The runtime failures associated with these bugs were fairly well hidden because you usually have to go through java to encounter them: scala doesn't pay that much attention to generic signatures, so they can be wrong and scala might still generate correct code. But java is not so lucky. Bug #1) During mixin composition, classes which extend traits receive forwarders to the implementations. An attempt was made to give these the correct info (in method "cloneBeforeErasure") but it was prone to giving the wrong answer, because: the key attribute which the forwarder must capture is what the underlying method will erase to *where the implementation is*, not how it appears to the class which contains it. That means the signature of the forwarder must be no more precise than the signature of the inherited implementation unless additional measures will be taken. This subtle difference will put on an unsubtle show for you in test run/t3452.scala. trait C[T] trait Search[M] { def search(input: M): C[Int] = null } object StringSearch extends Search[String] { } StringSearch.search("test"); // java // java.lang.NoSuchMethodError: StringSearch.search(Ljava/lang/String;)LC; The principled thing to do here would be to create a pair of methods in the host class: a mixin forwarder with the erased signature `(String)C[Int]`, and a bridge method with the same erased signature as the trait interface facet. But, this turns out to be pretty hard to retrofit onto the current setup of Mixin and Erasure, mostly due to the fact that mixin happens after erasure which has already taken care of bridging. For a future, release, we should try to move all bridging after mixin, and pursue this approach. But for now, what can we do about `LinkageError`s for Java clients? This commit simply checks if the pre-erasure method signature that we generate for the trait forward erases identically to that of the interface method. If so, we can be precise. If not, we emit the erased signature as the generic signature. Bug #2) The same principle is at work, at a different location. During genjvm, objects without declared companion classes are given static forwarders in the corresponding class, e.g. object Foo { def bar = 5 } which creates these classes (taking minor liberties): class Foo$ { static val MODULE$ = new Foo$ ; def bar = 5 } class Foo { static def bar = Foo$.MODULE$.bar } In generating these, genjvm circumvented the usual process whereby one creates a symbol and gives it an info, preferring to target the bytecode directly. However generic signatures are calculated from symbol info (in this case reusing the info from the module class.) Lacking even the attempt which was being made in mixin to "clone before erasure", we would have runtime failures of this kind: abstract class Foo { type T def f(x: T): List[T] = List() } object Bar extends Foo { type T = String } Bar.f(""); // java // java.lang.NoSuchMethodError: Bar.f(Ljava/lang/String;)Lscala/collection/immutable/List; Before/after this commit: < signature f (Ljava/lang/String;)Lscala/collection/immutable/List<Ljava/lang/String;>; --- > signature f (Ljava/lang/Object;)Lscala/collection/immutable/List<Ljava/lang/Object;>; This takes the warning count for compiling collections under `-Ycheck:jvm` from 1521 to 26.

As per discussion at https://groups.google.com/forum/#!topic/scala-internals/nf_ooEBn6-k, this commit introduces the new c.enclosingOwner API that is going to serve two purposes: 1) provide a better controlled alternative to c.enclosingTree, 2) enable low-level tinkering with owner chains without having to cast to compiler internals. This solution is not ideal, because: 1) symbols are much more than I would like to expose about enclosing lexical contexts (after the aforementioned discussion I’m no longer completely sure whether exposing nothing is the right thing to do, but exposing symbol completers is definitely something that should be avoided), 2) we shouldn’t have to do that low-level stuff in the first place. However, let’s face the facts. This change represents both an improvement over the state of the art wrt #1 and a long-awaited capability wrt #2. I think this pretty much warrants its place in trunk in the spirit of gradual, evolutionary development of reflection API.

SI-7937 In for, semi before guard never required

Under `-Ydelambdafy:method`, a public, static accessor method is created to expose the private method containing the body of the lambda. Currently this accessor method has its parameters in the same order structure as those of the lambda body method. What is this order? There are three categories of parameters: 1. lambda parameters 2. captured parameters (added by lambdalift) 3. self parameters (added to lambda bodies that end up in trait impl classes by mixin, and added unconditionally to the static accessor method.) These are currently emitted in order #3, #1, #2. Here are examples of the current behaviour: BEFORE (trait): ``` % cat sandbox/test.scala && scalac-hash v2.11.5 -Ydelambdafy:method sandbox/test.scala && javap -private -classpath . 'Test$class' trait Member; class Capture; trait LambdaParam trait Test { def member: Member def foo { val local = new Capture (arg: LambdaParam) => "" + arg + member + local } } Compiled from "test.scala" public abstract class Test$class { public static void foo(Test); private static final java.lang.String $anonfun$1(Test, LambdaParam, Capture); public static void $init$(Test); public static final java.lang.String accessor$1(Test, LambdaParam, Capture); } ``` BEFORE (class): ``` % cat sandbox/test.scala && scalac-hash v2.11.5 -Ydelambdafy:method sandbox/test.scala && javap -private -classpath . Test trait Member; class Capture; trait LambdaParam abstract class Test { def member: Member def foo { val local = new Capture (arg: LambdaParam) => "" + arg + member + local } } Compiled from "test.scala" public abstract class Test { public abstract Member member(); public void foo(); private final java.lang.String $anonfun$1(LambdaParam, Capture); public Test(); public static final java.lang.String accessor$1(Test, LambdaParam, Capture); } ``` Contrasting the class case with Java: ``` % cat sandbox/Test.java && javac -d . sandbox/Test.java && javap -private -classpath . Test public abstract class Test { public static class Member {}; public static class Capture {}; public static class LambaParam {}; public static interface I { public abstract Object c(LambaParam arg); } public abstract Member member(); public void test() { Capture local = new Capture(); I i1 = (LambaParam arg) -> "" + member() + local; } } Compiled from "Test.java" public abstract class Test { public Test(); public abstract Test$Member member(); public void test(); private java.lang.Object lambda$test$0(Test$Capture, Test$LambaParam); } ``` We can see that in Java 8 lambda parameters come after captures. If we want to use Java's LambdaMetafactory to spin up our anoymous FunctionN subclasses on the fly, our ordering must change. I can see three options for change: 1. Adjust `LambdaLift` to always prepend captured parameters, rather than appending them. I think we could leave `Mixin` as it is, it already prepends the self parameter. This would result a parameter ordering, in terms of the list above: #3, #2, #1. 2. More conservatively, do this just for methods known to hold lambda bodies. This might avoid needlessly breaking code that has come to depend on our binary encoding. 3. Adjust the parameters of the accessor method only. The body of this method can permute params before calling the lambda body method. This commit implements option #2. In also prototyped #1, and found it worked so long as I limited it to non-constructors, to sidestep the need to make corresponding changes elsewhere in the compiler to avoid the crasher shown in the enclosed test case, which was minimized from a bootstrap failure from an earlier a version of this patch. We would need to defer option #1 to 2.12 in any case, as some of these lifted methods are publicied by the optimizer, and we must leave the signatures alone to comply with MiMa. I've included a test that shows this in all in action. However, that is currently disabled, as we don't have a partest category for tests that require Java 8.

The log messages intented to chronicle implicit search were always being filtered out by virtue of the fact that the the tree passed to `printTyping` was already typed, (e.g. with an implicit MethodType.) This commit enabled printing in this case, although it still filters out trees that are deemed unfit for typer tracing, such as `()`. In the context of implicit search, this happens to filter out the noise of: ``` | | | [search #2] start `()`, searching for adaptation to pt=Unit => Foo[Int,Int] (silent: value <local Test> in Test) implicits disabled | | | [search #3] start `()`, searching for adaptation to pt=(=> Unit) => Foo[Int,Int] (silent: value <local Test> in Test) implicits disabled | | | \-> <error> ``` ... which I think is desirable. The motivation for this fix was to better display the interaction between implicit search and type inference. For instance: ``` class Foo[A, B] class Test { implicit val f: Foo[Int, String] = ??? def t[A, B](a: A)(implicit f: Foo[A, B]) = ??? t(1) } ``` ```` % scalac -Ytyper-debug sandbox/instantiate.scala ... | |-- t(1) BYVALmode-EXPRmode (site: value <local Test> in Test) | | |-- t BYVALmode-EXPRmode-FUNmode-POLYmode (silent: value <local Test> in Test) | | | [adapt] [A, B](a: A)(implicit f: Foo[A,B])Nothing adapted to [A, B](a: A)(implicit f: Foo[A,B])Nothing | | | \-> (a: A)(implicit f: Foo[A,B])Nothing | | |-- 1 BYVALmode-EXPRmode-POLYmode (site: value <local Test> in Test) | | | \-> Int(1) | | solving for (A: ?A, B: ?B) | | solving for (B: ?B) | | [search #1] start `[A, B](a: A)(implicit f: Foo[A,B])Nothing` inferring type B, searching for adaptation to pt=Foo[Int,B] (silent: value <local Test> in Test) implicits disabled | | [search #1] considering f | | [adapt] f adapted to => Foo[Int,String] based on pt Foo[Int,B] | | [search #1] solve tvars=?B, tvars.constr= >: String <: String | | solving for (B: ?B) | | [search #1] success inferred value of type Foo[Int,=?String] is SearchResult(Test.this.f, TreeTypeSubstituter(List(type B),List(String))) | | |-- [A, B](a: A)(implicit f: Foo[A,B])Nothing BYVALmode-EXPRmode (site: value <local Test> in Test) | | | \-> Nothing | | [adapt] [A, B](a: A)(implicit f: Foo[A,B])Nothing adapted to [A, B](a: A)(implicit f: Foo[A,B])Nothing | | \-> Nothing ```

Manually tested with: ``` % cat sandbox/test.scala package p { object X { def f(i: Int) = ??? ; def f(s: String) = ??? } object Main { val res = X.f(3.14) } } % qscalac -Ytyper-debug sandbox/test.scala |-- p EXPRmode-POLYmode-QUALmode (site: package <root>) | \-> p.type |-- object X BYVALmode-EXPRmode (site: package p) | |-- super EXPRmode-POLYmode-QUALmode (silent: <init> in X) | | |-- this EXPRmode (silent: <init> in X) | | | \-> p.X.type | | \-> p.X.type | |-- def f BYVALmode-EXPRmode (site: object X) | | |-- $qmark$qmark$qmark EXPRmode (site: method f in X) | | | \-> Nothing | | |-- Int TYPEmode (site: value i in X) | | | \-> Int | | |-- Int TYPEmode (site: value i in X) | | | \-> Int | | \-> [def f] (i: Int)Nothing | |-- def f BYVALmode-EXPRmode (site: object X) | | |-- $qmark$qmark$qmark EXPRmode (site: method f in X) | | | \-> Nothing | | |-- String TYPEmode (site: value s in X) | | | [adapt] String is now a TypeTree(String) | | | \-> String | | |-- String TYPEmode (site: value s in X) | | | [adapt] String is now a TypeTree(String) | | | \-> String | | \-> [def f] (s: String)Nothing | \-> [object X] p.X.type |-- object Main BYVALmode-EXPRmode (site: package p) | |-- X.f(3.14) EXPRmode (site: value res in Main) | | |-- X.f BYVALmode-EXPRmode-FUNmode-POLYmode (silent: value res in Main) | | | |-- X EXPRmode-POLYmode-QUALmode (silent: value res in Main) | | | | \-> p.X.type | | | \-> (s: String)Nothing <and> (i: Int)Nothing | | |-- 3.14 BYVALmode-EXPRmode (silent: value res in Main) | | | \-> Double(3.14) | | [search #1] start `<?>`, searching for adaptation to pt=Double => String (silent: value res in Main) implicits disabled | | [search #2] start `<?>`, searching for adaptation to pt=(=> Double) => String (silent: value res in Main) implicits disabled | | [search #3] start `<?>`, searching for adaptation to pt=Double => Int (silent: value res in Main) implicits disabled | | 1 implicits in companion scope | | [search #4] start `<?>`, searching for adaptation to pt=(=> Double) => Int (silent: value res in Main) implicits disabled | | 1 implicits in companion scope | | second try: <error> and 3.14 | | [search #5] start `p.X.type`, searching for adaptation to pt=p.X.type => ?{def f(x$1: ? >: Double(3.14)): ?} (silent: value res in Main) implicits disabled | | [search #6] start `p.X.type`, searching for adaptation to pt=(=> p.X.type) => ?{def f(x$1: ? >: Double(3.14)): ?} (silent: value res in Main) implicits disabled sandbox/test.scala:4: error: overloaded method value f with alternatives: (s: String)Nothing <and> (i: Int)Nothing cannot be applied to (Double) val res = X.f(3.14) ^ ```

little documentation fix

Moved `Ev` from a generic type parameter to a member type in CIPT.

Before: ``` implicitly[Foo[String]] BYVALmode-EXPRmode (site: value <local Test> in Test) |-- implicitly BYVALmode-EXPRmode-FUNmode-POLYmode-TAPPmode (site: value <local Test> in Test) | \-> [T](implicit e: T)T |-- Foo[String] TYPEmode (site: value <local Test> in Test) | |-- String TYPEmode (site: value <local Test> in Test) | | [adapt] String is now a TypeTree(String) | | \-> String | \-> Foo[String] [search #1] start `[T](implicit e: T)T`, searching for adaptation to pt=Foo[String] (silent: value <local Test> in Test) implicits disabled [search #1] considering Foo.javaEnumFoo solving for (T: ?T) [adapt] [T]=> Foo[T] adapted to [T]=> Foo[T] based on pt Foo[String] [search #1] success inferred value of type Foo[String] is SearchResult(Foo.javaEnumFoo[String], ) |-- [T](implicit e: T)T BYVALmode-EXPRmode (site: value <local Test> in Test) | \-> Foo[String] [adapt] [T](implicit e: T)T adapted to [T](implicit e: T)T ``` After: ``` implicitly[Foo[String]] BYVALmode-EXPRmode (site: value <local Test> in Test) |-- implicitly BYVALmode-EXPRmode-FUNmode-POLYmode-TAPPmode (site: value <local Test> in Test) | \-> [T](implicit e: T)T |-- Foo[String] TYPEmode (site: value <local Test> in Test) | |-- String TYPEmode (site: value <local Test> in Test) | | [adapt] String is now a TypeTree(String) | | \-> String | \-> Foo[String] [search #1] start `[T](implicit e: T)T`, searching for adaptation to pt=Foo[String] (silent: value <local Test> in Test) implicits disabled [search #1] considering Foo.stringFoo [search #1] success inferred value of type Foo[String] is SearchResult(Foo.stringFoo, ) |-- [T](implicit e: T)T BYVALmode-EXPRmode (site: value <local Test> in Test) | \-> Foo[String] [adapt] [T](implicit e: T)T adapted to [T](implicit e: T)T \-> Foo[String] ```

Before: ``` implicitly[Foo[String]] BYVALmode-EXPRmode (site: value <local Test> in Test) |-- implicitly BYVALmode-EXPRmode-FUNmode-POLYmode-TAPPmode (site: value <local Test> in Test) | \-> [T](implicit e: T)T |-- Foo[String] TYPEmode (site: value <local Test> in Test) | |-- String TYPEmode (site: value <local Test> in Test) | | [adapt] String is now a TypeTree(String) | | \-> String | \-> Foo[String] [search #1] start `[T](implicit e: T)T`, searching for adaptation to pt=Foo[String] (silent: value <local Test> in Test) implicits disabled [search #1] considering Foo.javaEnumFoo solving for (T: ?T) [adapt] [T]=> Foo[T] adapted to [T]=> Foo[T] based on pt Foo[String] [search #1] success inferred value of type Foo[String] is SearchResult(Foo.javaEnumFoo[String], ) |-- [T](implicit e: T)T BYVALmode-EXPRmode (site: value <local Test> in Test) | \-> Foo[String] [adapt] [T](implicit e: T)T adapted to [T](implicit e: T)T ``` After: ``` implicitly[Foo[String]] BYVALmode-EXPRmode (site: value <local Test> in Test) |-- implicitly BYVALmode-EXPRmode-FUNmode-POLYmode-TAPPmode (site: value <local Test> in Test) | \-> [T](implicit e: T)T |-- Foo[String] TYPEmode (site: value <local Test> in Test) | |-- String TYPEmode (site: value <local Test> in Test) | | [adapt] String is now a TypeTree(String) | | \-> String | \-> Foo[String] [search #1] start `[T](implicit e: T)T`, searching for adaptation to pt=Foo[String] (silent: value <local Test> in Test) implicits disabled [search #1] considering Foo.stringFoo [search #1] success inferred value of type Foo[String] is SearchResult(Foo.stringFoo, ) |-- [T](implicit e: T)T BYVALmode-EXPRmode (site: value <local Test> in Test) | \-> Foo[String] [adapt] [T](implicit e: T)T adapted to [T](implicit e: T)T \-> Foo[String] ``` (cherry picked from commit 68799b9)

phaller added 2 commits January 26, 2012 12:55

Refactor source context generation in implicits

69f8e42

Move code for updating search result for SourceLocation/SourceContext…

1ef4ad2

… into SourceContextUtils

adriaanm added a commit that referenced this pull request Feb 2, 2012

Merge pull request #1 from phaller/topic/virt_sourcecontext

d0cf9c3

refactoring sourcecontext support into its own trait

adriaanm merged commit d0cf9c3 into adriaanm:topic/virt Feb 2, 2012

adriaanm pushed a commit that referenced this pull request Feb 2, 2012

Merge pull request #1 from lpereir4/avl

bdb3fc8

AvlTree performance improvements

adriaanm pushed a commit that referenced this pull request Feb 20, 2013

Merge pull request #1 from retronym/topic/annotatedRetyping-test

9397620

Additional test case for Lukas' fix to annotated originals.

adriaanm pushed a commit that referenced this pull request Sep 19, 2013

Merge pull request #1 from retronym/ticket/7767-test

aa1bff8

Test case for SI-7767

adriaanm added a commit that referenced this pull request Mar 27, 2014

Merge pull request #1 from som-snytt/issue/7937

be3b73e

SI-7937 In for, semi before guard never required

SethTisue pushed a commit that referenced this pull request Apr 25, 2018

Merge pull request #1 from esarbe/master

f5329f9

little documentation fix

SethTisue pushed a commit that referenced this pull request Apr 25, 2018

Merge pull request #1 from Ichoran/wip/cc2-cipt-internal-ev

7c4a776

Moved `Ev` from a generic type parameter to a member type in CIPT.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Topic/virt sourcecontext #1

Topic/virt sourcecontext #1

phaller commented Jan 27, 2012

Topic/virt sourcecontext #1

Topic/virt sourcecontext #1

Conversation

phaller commented Jan 27, 2012