[sammy] eta-expansion, overloading, existentials #4101

adriaanm · 2014-11-06T11:06:09Z

Review by @retronym, please.

While playing with Java 8 Streams from the repl,
I realized we weren't eta-expanding,
nor resolving overloading properly for SAMs.

Also, the way Java uses wildcards to represent use-site variance
stresses type inference (since it results in existentials).

Introduce wildcardExtrapolation to simplify the resulting types
(without losing precision): wildcardExtrapolation(tp) =:= tp.

For example, the MethodType given by def bla(x: (_ >: String)): (_ <: Int)
is both a subtype and a supertype of def bla(x: String): Int.

Translating http://winterbe.com/posts/2014/07/31/java8-stream-tutorial-examples/ into Scala:

scala> import java.util.Arrays
scala> import java.util.stream.Stream
scala> import java.util.stream.IntStream
scala> val myList = Arrays.asList("a1", "a2", "b1", "c2", "c1")
myList: java.util.List[String] = [a1, a2, b1, c2, c1]

scala> myList.stream.filter(_.startsWith("c")).map(_.toUpperCase).sorted.forEach(println)
C1
C2

scala> myList.stream.filter(_.startsWith("c")).map(_.toUpperCase).sorted
res8: java.util.stream.Stream[?0] = java.util.stream.SortedOps$OfRef@133e7789

scala> Arrays.asList("a1", "a2", "a3").stream.findFirst.ifPresent(println)
a1

scala> Stream.of("a1", "a2", "a3").findFirst.ifPresent(println)
a1

scala> IntStream.range(1, 4).forEach(println)
<console>:37: error: object creation impossible, since method accept in trait IntConsumer of type (x$1: Int)Unit is not defined
(Note that Int does not match Any: class Int in package scala is a subclass of class Any in package scala, but method parameter types must match exactly.)
              IntStream.range(1, 4).forEach(println)
                                            ^

scala> IntStream.range(1, 4).forEach(println(_: Int)) // TODO: can we avoid this annotation?
1
2
3

scala> Arrays.stream(Array(1, 2, 3)).map(n => 2 * n + 1).average.ifPresent(println(_: Double))
5.0

scala> Stream.of("a1", "a2", "a3").map(_.substring(1)).mapToInt(_.parseInt).max.ifPresent(println(_: Int)) // whoops!
ReplGlobal.abort: Unknown type: <error>, <error> [class scala.reflect.internal.Types$ErrorType$, class scala.reflect.internal.Types$ErrorType$] TypeRef? false
error: Unknown type: <error>, <error> [class scala.reflect.internal.Types$ErrorType$, class scala.reflect.internal.Types$ErrorType$] TypeRef? false
scala.reflect.internal.FatalError: Unknown type: <error>, <error> [class scala.reflect.internal.Types$ErrorType$, class scala.reflect.internal.Types$ErrorType$] TypeRef? false
    at scala.reflect.internal.Reporting$class.abort(Reporting.scala:59)

scala> IntStream.range(1, 4).mapToObj(i => "a" + i).forEach(println)
a1
a2
a3

adriaanm · 2014-11-06T11:07:10Z

/cc FYI, @Ichoran

adriaanm · 2014-11-06T11:07:28Z

I'll investigate the failures in the REPL transcript later. Need to get back to my talk :)

gourlaysama · 2014-11-06T11:29:31Z

👍 Yay! Overloading is the thing that prevented SAMs from being really useful when calling into java.

BTW there was a ticket for that: SI-8310

adriaanm · 2014-11-06T12:56:00Z

Ugh, the jenkins node I added is running an outdated version of java.

adriaanm · 2014-11-06T12:56:25Z

Thanks for the pointer @gourlaysama! I'll include the test case -- it passes now.

retronym · 2014-11-06T13:09:46Z

I offlined them about 15 mins ago.

Playing with Java 8 Streams from the repl showed we weren't eta-expanding, nor resolving overloading for SAMs. Also, the way Java uses wildcards to represent use-site variance stresses type inference past its bendiness point (due to excessive existentials). I introduce `wildcardExtrapolation` to simplify the resulting types (without losing precision): `wildcardExtrapolation(tp) =:= tp`. For example, the `MethodType` given by `def bla(x: (_ >: String)): (_ <: Int)` is both a subtype and a supertype of `def bla(x: String): Int`. Translating http://winterbe.com/posts/2014/07/31/java8-stream-tutorial-examples/ into Scala shows most of this works, though we have some more work to do (see near the end). ``` scala> import java.util.Arrays scala> import java.util.stream.Stream scala> import java.util.stream.IntStream scala> val myList = Arrays.asList("a1", "a2", "b1", "c2", "c1") myList: java.util.List[String] = [a1, a2, b1, c2, c1] scala> myList.stream.filter(_.startsWith("c")).map(_.toUpperCase).sorted.forEach(println) C1 C2 scala> myList.stream.filter(_.startsWith("c")).map(_.toUpperCase).sorted res8: java.util.stream.Stream[?0] = java.util.stream.SortedOps$OfRef@133e7789 scala> Arrays.asList("a1", "a2", "a3").stream.findFirst.ifPresent(println) a1 scala> Stream.of("a1", "a2", "a3").findFirst.ifPresent(println) a1 scala> IntStream.range(1, 4).forEach(println) <console>:37: error: object creation impossible, since method accept in trait IntConsumer of type (x$1: Int)Unit is not defined (Note that Int does not match Any: class Int in package scala is a subclass of class Any in package scala, but method parameter types must match exactly.) IntStream.range(1, 4).forEach(println) ^ scala> IntStream.range(1, 4).forEach(println(_: Int)) // TODO: can we avoid this annotation? 1 2 3 scala> Arrays.stream(Array(1, 2, 3)).map(n => 2 * n + 1).average.ifPresent(println(_: Double)) 5.0 scala> Stream.of("a1", "a2", "a3").map(_.substring(1)).mapToInt(_.parseInt).max.ifPresent(println(_: Int)) // whoops! ReplGlobal.abort: Unknown type: <error>, <error> [class scala.reflect.internal.Types$ErrorType$, class scala.reflect.internal.Types$ErrorType$] TypeRef? false error: Unknown type: <error>, <error> [class scala.reflect.internal.Types$ErrorType$, class scala.reflect.internal.Types$ErrorType$] TypeRef? false scala.reflect.internal.FatalError: Unknown type: <error>, <error> [class scala.reflect.internal.Types$ErrorType$, class scala.reflect.internal.Types$ErrorType$] TypeRef? false at scala.reflect.internal.Reporting$class.abort(Reporting.scala:59) scala> IntStream.range(1, 4).mapToObj(i => "a" + i).forEach(println) a1 a2 a3 ```

adriaanm · 2014-11-06T13:20:14Z

Thanks -- I push --forced to address review feedback (include SI-8310 test case).

adriaanm · 2014-11-06T16:56:25Z

Need to update that check file. The new error message is correct, though suboptimal...

retronym · 2014-11-06T23:27:39Z

src/compiler/scala/tools/nsc/typechecker/Infer.scala

      val tp1 = normalize(tp)

      (    (tp1 weak_<:< pt)
        || isCoercible(tp1, pt)
        || isCompatibleByName(tp, pt)
+        || isCompatibleSam(tp, pt)


I don't see where we predicate this on -Xexperimental. I believe we should. I'd also be interested in performance impact of this; we should have a fast path for non-SAMs.

Oh wait, I see samOf checks the settings. That will do.

I figured it's easier to flip the switch if it's only tested in one central spot.

Don't naively derive types for the single method's signature from the provided function's type, as it may be a subtype of the method's MethodType. Instead, once the sam class type is fully defined, determine the sam's info as seen from the class's type, and use those to generate the correct override. ``` scala> Arrays.stream(Array(1, 2, 3)).map(n => 2 * n + 1).average.ifPresent(println) 5.0 scala> IntStream.range(1, 4).forEach(println) 1 2 3 ``` Also, minimal error reporting Can't figure out how to do it properly, but some reporting is better than crashing. Right? Test case that illustrates necessity of the clumsy stop gap `if (block exists (_.isErroneous))` enclosed as `sammy_error_exist_no_crash` added TODO for repeated and by-name params

adriaanm · 2014-11-07T11:16:55Z

BTW, https://gist.github.com/adriaanm/892d6063dd485d7dd221

Generate correct trees to refer to repeated params using `gen.paramToArg`. Based on retronym's review feedback.

retronym · 2014-11-09T01:12:12Z

LGTM

[sammy] eta-expansion, overloading, existentials

VladUreche · 2015-01-14T20:59:18Z

src/compiler/scala/tools/nsc/typechecker/Typers.scala

@@ -2701,6 +2725,22 @@ trait Typers extends Adaptations with Tags with TypersTracking with PatternTyper
     * However T must be fully defined before we type the instantiation, as it'll end up as a parent type,
     * which must be fully defined. Would be nice to have some kind of mechanism to insert type vars in a block of code,
     * and have the instantiation of the first occurrence propagate to the rest of the block.
+     *
+     * TODO: by-name params


@adriaanm, by this you mean automatically applying the SAM implementation's abstract method?

Not sure what you're referring to, but I meant that f(println("!")) should behave the same as f.accept(println("!")) below.

Got it now, thanks @adriaanm!

VladUreche · 2015-01-14T21:03:01Z

Cool patch! I'm super excited about this making its way to 2.11.5 👍

In some cases existential bounds can be simplified without losing precision. For example: trait Blargle[T] { def compare(a: T, b: T): Int } trait Test { def foo(a: Blargle[_ >: String]): Int } can be simplified to: trait Test { def foo(a: Blargle[String]): Int } see: scala/scala#4101 #SCL8956 Fixed

scala-jenkins added this to the 2.11.5 milestone Nov 6, 2014

scala-jenkins added the needs-attention label Nov 6, 2014

adriaanm force-pushed the sam-ex branch from 50990e4 to 9c36a76 Compare November 6, 2014 13:19

retronym reviewed Nov 6, 2014
View reviewed changes

adriaanm force-pushed the sam-ex branch from f6f8bdf to 333c2ed Compare November 7, 2014 10:59

adriaanm force-pushed the sam-ex branch from 333c2ed to a34be74 Compare November 7, 2014 11:00

scala-jenkins added tested and removed needs-attention labels Nov 7, 2014

adriaanm force-pushed the sam-ex branch from fb507a4 to 0d7ae7f Compare November 7, 2014 16:50

[sammy] support repeated params

cbca494

Generate correct trees to refer to repeated params using `gen.paramToArg`. Based on retronym's review feedback.

adriaanm force-pushed the sam-ex branch from 0d7ae7f to cbca494 Compare November 7, 2014 16:51

scala-jenkins added tested and removed tested labels Nov 7, 2014

scala-jenkins added the reviewed label Nov 9, 2014

lrytz added a commit that referenced this pull request Nov 10, 2014

Merge pull request #4101 from adriaanm/sam-ex

5a7875f

[sammy] eta-expansion, overloading, existentials

lrytz merged commit 5a7875f into scala:2.11.x Nov 10, 2014

VladUreche reviewed Jan 14, 2015
View reviewed changes

ilinum mentioned this pull request Jul 29, 2015

SAM Fixes JetBrains/intellij-scala#189

Merged

adriaanm deleted the sam-ex branch August 5, 2015 01:31

scabug mentioned this pull request Apr 7, 2017

sams could play nicer with overloading scala/bug#8310

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[sammy] eta-expansion, overloading, existentials #4101

[sammy] eta-expansion, overloading, existentials #4101

adriaanm commented Nov 6, 2014

adriaanm commented Nov 6, 2014

adriaanm commented Nov 6, 2014

gourlaysama commented Nov 6, 2014

adriaanm commented Nov 6, 2014

adriaanm commented Nov 6, 2014

retronym commented Nov 6, 2014

adriaanm commented Nov 6, 2014

adriaanm commented Nov 6, 2014

retronym Nov 6, 2014

retronym Nov 6, 2014

adriaanm Nov 7, 2014

adriaanm commented Nov 7, 2014

retronym commented Nov 9, 2014

VladUreche Jan 14, 2015

adriaanm Jan 14, 2015

VladUreche Jan 14, 2015

VladUreche commented Jan 14, 2015

[sammy] eta-expansion, overloading, existentials #4101

[sammy] eta-expansion, overloading, existentials #4101

Conversation

adriaanm commented Nov 6, 2014

adriaanm commented Nov 6, 2014

adriaanm commented Nov 6, 2014

gourlaysama commented Nov 6, 2014

adriaanm commented Nov 6, 2014

adriaanm commented Nov 6, 2014

retronym commented Nov 6, 2014

adriaanm commented Nov 6, 2014

adriaanm commented Nov 6, 2014

retronym Nov 6, 2014

Choose a reason for hiding this comment

retronym Nov 6, 2014

Choose a reason for hiding this comment

adriaanm Nov 7, 2014

Choose a reason for hiding this comment

adriaanm commented Nov 7, 2014

retronym commented Nov 9, 2014

VladUreche Jan 14, 2015

Choose a reason for hiding this comment

adriaanm Jan 14, 2015

Choose a reason for hiding this comment

VladUreche Jan 14, 2015

Choose a reason for hiding this comment

VladUreche commented Jan 14, 2015