Use default methods in our trait encoding

[retronym]

Java 8 allows interfaces to contain default implementations of methods. This is done by
removing a restriction in the bytecode, interface methods may now have a code attribute.

Current trait encoding

• typically, a concrete method in a Scala trait results in:
  • in the interface: an deferred method in the interface
  • in the impl class: an impl method containing the body of the method, taking $this as an extra parameter
  • in each subclass: a forwarder method that implements the interface method and forwards to the implementation class

Proposed encoding

• Do away with the impl methods, instead leaving the code in a default method of the interface method
• Optional: omit forwarder methods in subclasses if the JVM would link an invokeinterface
  to the same method as the forwarder would target.
• when forwarders are still used, or when we encode super[Trait].method, we can use invokespecial
  to call the default implementation of interface method.

Benefits

• making it easier to extends traits from Java, including with Java lambdas
  • which will obviate the layer of JFunctionN interfaces initially used to back Scala 2.12 lambdas
• adding a concrete method to a trait can be a binary compatible without requiring subclasses to recompile
• (if we omit forwarders) reduced bytecode size
• (if we omit forwarders) avoid the JIT-hostile code shapes of the artificially megamorphic calls, where all implementations of the methods are just forwarders to the same implementation method.

Non Benefits

• fields in traits still require "intimate" recompilation of trait and subclass.

Steps

• Update GenBCode to enable emission of default methods #38 Enable emission of defaults
• Bridge or omit troublesome additional abstract methods in specialized traits #36 Handle interaction with specialization
• Remove AddInterfaces, impl classes, and redirect references to impl methods to default methods #37 Do away with impl classes

[retronym]

Omitting forwarders has a benefit for binary compatibility in this situation. I think it should be our preferred approach.

% echo 'trait A { def m = "a"}; 
        trait B extends A; 
        object Test extends App with B { println(m) }' > a.scala && scalac a.scala 
% echo 'trait B extends A { override def m = "b" }' > a.scala && scalac a.scala
% scala Test
a

% echo ':javap Test$#m' | scala
Welcome to Scala version 2.11.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_51).
Type in expressions to have them evaluated.
Type :help for more information.

scala> :javap Test$#m
  public java.lang.String m();
    descriptor: ()Ljava/lang/String;
    flags: ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: aload_0
         1: invokestatic  #30                 // Method A$class.m:(LA;)Ljava/lang/String;
         4: areturn
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       5     0  this   LTest$;
      LineNumberTable:
        line 1: 0

scala> :quit

% echo 'object Test extends App with B { println(m) }' > a.scala; scalac a.scala; scala Test
b

[adriaanm]

+1, note that even in the scenario where B does not extend A, but where Test extends A with B (or B with A), similar benefits hold, though the dependency tracker should trigger fuller recompiles

[SethTisue]

as I said on Slack, the scenario I'm concerned about is:

suppose trait T1 { def foo = 1 }; trait T2; class C extends T1 with T2. in C, if we don’t generate a forwarder that invokespecials T1.foo, then (new C).foo will still work thanks to the semantics of default methods. now suppose T2 is in a library, and suppose that we upgrade to a newer version of the library in which T2 has been redefined to trait T2 { def foo = 2}. if we don’t recompile anything, the behavior of (new C).foo is now undefined and might produce either 1 or 2.

whoa, you say, "undefined"? yes, JSR 335 says:

if any superinterface of C declares a method with the name and descriptor specified by the method reference that has set neither its ACC_PRIVATE flag nor its ACC_STATIC flag, one of these is arbitrarily chosen and method lookup succeeds

[SethTisue]

Adriaan found this, which may indicate this is not so undefined and indeterministic after all: https://jvilk.com/blog/java-8-specification-bug/

[dragos]

How does this handle fields in traits? Are default methods going to take a $this parameter, like impl methods do now?

[retronym]

Fields will work as today. Default methods on interfaces take the this parameter implicitly like regular instance methods, the compiler does not need to add the parameter.

[DarkDimius]

At least in dotty they do not.
Default methods can access 'this' of the interface, and call fetters and
setters through it.

On 14 August 2015 12:32:21 Iulian Dragos notifications@github.com wrote:

How does this handle fields in traits? Are default methods going to take a
$this parameter, like impl methods do now?

---

Reply to this email directly or view it on GitHub:
#35 (comment)

[retronym]

Copying Adriaan's mail here.

Now that work on the second main feature of 2.12 is under way -- compiling traits to interfaces -- a quick update. Thanks to the Dotty compiler, where this is already implemented, we have confidence that we can compile all Scala trait to interfaces!

Because Scala member lookup semantics are different from Java's, we will have to generate a fair amount of synthetic code in classes that extend traits to make sure we get Scala's linearization semantics and not Java's "default methods are not called when an instance method is available". See the linked gist for an example.

Most of the current trait encoding carries over directly. The most exciting simplificaiton is that we no longer need the trait's implementation class (T$class)! As it was mostly a holder for the bodies of concrete trait methods, which become default methods in the interface. (We'll need to deal with initialization methods for lazy vals and objects slightly differently, but no biggy.)

When linearization order is known (at the class that inherits traits), we can add fields, implement super/outer/field/object/lazy val accessors and enforce the linearization order.

For super accessors, for example, the meaning of a super call super.m in a trait T depends on the order of parents for the class that extends T, so it is compiled to a call to the abstract method T$$super$m defined in T, which can only be implemented in the class that inherits T.

The linearization order is codified by overriding every method that has an implementation in one of the base classes/traits, with a method that calls the correct super-method. This means binary compatibility suffers in a similar way as the current trait encoding, although you will be able to introduce new methods in traits without incurring a linkage error, the runtime behavior will only be correct if there's no overriding instance method (see the gist above).

[retronym]

Follow on tickets: #98 #59