Skip to content
This repository was archived by the owner on Sep 1, 2020. It is now read-only.

Backport code from typesafe scala #80

Closed
wants to merge 40 commits into from

Conversation

folone
Copy link

@folone folone commented Oct 24, 2014

2.11.4 was just staged: https://groups.google.com/forum/#!topic/scala-internals/y890GcpjONE. Time to backport the changes?

@folone folone changed the title Backport code from mainline scala Backport code from typesafe scala Oct 24, 2014
@folone
Copy link
Author

folone commented Oct 24, 2014

@milessabin
Copy link
Member

+1

@non
Copy link

non commented Oct 24, 2014

It builds and passes tests, 👍

som-snytt and others added 26 commits October 28, 2014 10:18
People expect to change the class path midstream.

Let's disabuse them by removing the broken command.

The internals are deprecated.
The reset and replay commands take arbitrary command line args.
When settings args are supplied, the compiler is recreated.

For uniformity, the settings command performs only the usual
arg parsing: use -flag:true instead of +flag, and clearing a
setting is promoted to the command line, so that -Xlint: is not
an error but clears the flags.

```
scala> maqicode.Test main null
<console>:8: error: not found: value maqicode
              maqicode.Test main null
              ^

scala> :reset -classpath/a target/scala-2.11/sample_2.11-1.0.jar
Resetting interpreter state.
Forgetting all expression results and named terms: $intp

scala> maqicode.Test main null
Hello, world.

scala> val i = 42
i: Int = 42

scala> s"$i is the loneliest numbah."
res1: String = 42 is the loneliest numbah.

scala> :replay -classpath ""
Replaying: maqicode.Test main null
Hello, world.

Replaying: val i = 42
i: Int = 42

Replaying: s"$i is the loneliest numbah."
res1: String = 42 is the loneliest numbah.

scala> :replay -classpath/a ""
Replaying: maqicode.Test main null
<console>:8: error: not found: value maqicode
              maqicode.Test main null
              ^

Replaying: val i = 42
i: Int = 42

Replaying: s"$i is the loneliest numbah."
res1: String = 42 is the loneliest numbah.
```

Clearing a clearable setting:
```
scala> :reset -Xlint:missing-interpolator
Resetting interpreter state.

scala> { val i = 42 ; "$i is the loneliest numbah." }
<console>:8: warning: possible missing interpolator: detected interpolated identifier `$i`
              { val i = 42 ; "$i is the loneliest numbah." }
                             ^
res0: String = $i is the loneliest numbah.

scala> :reset -Xlint:
Resetting interpreter state.
Forgetting this session history:

{ val i = 42 ; "$i is the loneliest numbah." }

scala> { val i = 42 ; "$i is the loneliest numbah." }
res0: String = $i is the loneliest numbah.

```
* Errors are red
* Warnings are yellow
Added some documentation explaining what the role of `end` is.
Under load on Jenkins, we've been seeing:

```
% diff /localhome/jenkins/a/workspace/scala-nightly-auxjvm-2.12.x/jdk/jdk7/label/auxjvm/test/files/run/t4542-run.log /localhome/jenkins/a/workspace/scala-nightly-auxjvm-2.12.x/jdk/jdk7/label/auxjvm/test/files/run/t4542.check
@@ -2,75 +2,14 @@ Type in expressions to have them evaluated.
 Type :help for more information.

 scala> @deprecated("foooo", "ReplTest version 1.0-FINAL") class Foo() {
 java.util.concurrent.TimeoutException: Futures timed out after [60 seconds]
 	at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
 	at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
 	at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:95)
 	at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:95)
 	at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
 	at scala.concurrent.Await$.ready(package.scala:95)
 	at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:431)
 	at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:457)
 	at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:875)
```

This commit bumps the timeout up be a factor of ten to try to
restore that comforting green glow to https://scala-webapps.epfl.ch/jenkins/view/2.N.x
Given that we'll switch to GenBCode in 2.12, the test case
showing the bug is fixed under that option is all I plan to
offer for this bug.

The flags file contains `-Ynooptimize` to stay locked into
`GenBCode`.
Previously, abstract type members were allowed in objects only when inherited,
but not when declared directly. This inconsistency was not intended. In dotty,
abstract type members are allowed in values and represent existentials; so upon
discussion, it was decided to fix things to conform to dotty and allow such type
members. Adriaan also asked to keep rejecting abstract type members in methods
even though they would conceivably make sense.

Discussions happened on scala#3407, scala/scala-dist#127.
This code is improved from scala#3442, keeps closer to the current logic, and passes tests.

Existing tests that have been converted to `pos` tests show that
this works, and a new test has been added to show that local
aliases (ie term-owned) without a RHS are still rejected.
... which can be introduced by `memberType` for methods
with parameter types dependent on class type parameters.

Here's an example of such a type:

```
scala> class Bippy { trait Foo[A] }
defined class Bippy

scala> final class RichBippy[C <: Bippy with Singleton](val c1: C) {
     |   def g[A](x: A)(ev: c1.Foo[A]): Int = 2
     | }
defined class RichBippy

scala> :power
** Power User mode enabled - BEEP WHIR GYVE **
** :phase has been set to 'typer'.          **
** scala.tools.nsc._ has been imported      **
** global._, definitions._ also imported    **
** Try  :help, :vals, power.<tab>           **

scala> val g = typeOf[RichBippy[_]].member(TermName("g"))
g: $r.intp.global.Symbol = method g

scala> val c = new Bippy
c: Bippy = Bippy@92e2c93

scala> val memberType = typeOf[RichBippy[c.type]].memberType(g)
memberType: $r.intp.global.Type = ([A](x: A)(ev: _7.c1.Foo[A])Int) forSome { val _7: RichBippy[c.type] }

```

In this example, if we were to typecheck the selection
`new RichBippy[c.type].g` that existential type would be short lived.

Consider this approximation of `Typer#typedInternal`:

```scala
val tree1: Tree = typed1(tree, mode, ptWild)
val result = adapt(tree1, mode, ptPlugins, tree)
```

Given that `tree1.tpe` is not an overloaded, adapt will find its
way to:

```
  case tp if mode.typingExprNotLhs && isExistentialType(tp) =>
    adapt(tree setType tp.dealias.skolemizeExistential(context.owner, tree), mode, pt, original)
```

Which would open the existential as per:

```
scala> memberType.skolemizeExistential
res2: $r.intp.global.Type = [A](x: A)(ev: _7.c1.Foo[A])Int
```

However, if do have overloaded alternatives, as in the test case,
we have to remember to call `adapt` again *after* we have picked
the winning alternative.

We actually don't have a centralised place where overload resolution
occurs, as the process differs depending on the context of the
selection. (Are there explicit type arguments? Inferred type
arguments? Do we need to use the expected type to pick a winner?)

This commit finds the existing places that call adapt after
overloade resolution and routes those calls through a marker
method. It then adds one more call to this in `inferPolyAlternatives`,
which fixes the bug.
This reverts commit 8986ee4.

Scaladoc seems to work reliably for 2.11.x. We are using it in the IDE builds and haven't noticed any
flakiness, so we'd like to get reinstate this test.
This pattern of code is typically a bug:

    if (f(tp.typeSymbol)) {
       g(tp.typeArgs)
    }

Intead, one needs to take the base type of `tp` wrt `tp.typeSymbol`.

This commit does exactly that when formatting the `@implicitNotFound`
custom error message.

Patch found on the back of an envelope in the handwriting of @adriaanm
Fixes non-determinism within the DPLL algorithm and disallows
infeasible counter examples directly in the formula.

The function to compute all solutions was flawed and thus only
returned a subset of the solutions. The algorithm would stop too soon
and thus depending on the ordering of the symbols return more or less
solutions. I also added printing a warning when the search was stopped
because the max recursion depth was reached. This is very useful as an
explanation of spuriously failing regression tests, since less counter
examples might be reported. In such a case the recursion depth should
be set to infinite by adding `-Ypatmat-exhaust-depth off`.

The mapping of the solutions of the DPLL algorithm to counter examples
has been adapted to take the additional solutions from the
solver into account:

Consider for example `t8430.scala`:

```Scala
sealed trait CL3Literal
case object IntLit extends CL3Literal
case object CharLit extends CL3Literal
case object BooleanLit extends CL3Literal
case object UnitLit extends CL3Literal

sealed trait Tree
case class LetL(value: CL3Literal) extends Tree
case object LetP extends Tree
case object LetC extends Tree
case object LetF extends Tree

object Test {
  (tree: Tree) => tree match {case LetL(CharLit) => ??? }
}
```

This test contains 2 domains,  `IntLit, CharLit, ...` and `LetL, LetP, ...`,
the corresponding formula to check exhaustivity looks like:

```
    V1=LetC.type#13    \/ V1=LetF.type#14 \/    V1=LetL#11     \/  V1=LetP.type#15    /\
 V2=BooleanLit.type#16 \/  V2=CharLit#12  \/ V2=IntLit.type#17 \/ V2=UnitLit.type#18  /\
      -V1=LetL#11      \/ -V2=CharLit#12  \/                   \/
```

The first two lines assign a value of the domain to the scrutinee (and
the correponding member in case of `LetL`) and prohibit the counter
example `LetL(CharLit)` since it's covered by the pattern match. The
used Boolean encoding allows that scrutinee `V1` can be equal to
`LetC` and `LetF` at the same time and thus, during enumeration of all
possible solutions of the formula, such a solution will be found,
since only one literal needs to be set to true, to  satisfy that
clause. That means, if at least one of the literals of such a clause
was in the `unassigned` list of the DPLL procedure, we will get
solutions where the scrutinee is equal to more than one element of the
domain.

A remedy would be to add constraints that forbid both literals
to be true at the same time. His is infeasible for big domains (see
`pos/t8531.scala`), since we would have to add a quadratic number of
clauses (one clause for each pair in the domain). A much simpler
solution is to just filter the invalid results. Since both values for
`unassigned` literals are explored, we will eventually find a valid
counter example.
to initial implementation).

Assuming that the DPLL procedure does not run into max recursion
depth, that workaround is not needed anymore, since the non-
determinism has been fixed.
Let the AbstractFileClassLoader override just the usual suspects.

Normal delegation behavior should ensue.

That's instead of overriding `getResourceAsStream`, which was intended
that "The repl classloader now works more like you'd expect a classloader to."
(Workaround for "Don't know how to construct an URL for something which exists
only in memory.")

Also override `findResources` so that `getResources` does the obvious thing,
namely, return one iff `getResource` does.

The translating class loader for REPL only special-cases `foo.class`: as
a fallback, take `foo` as `$line42.$read$something$foo` and try that class file.
That's the use case for "works like you'd expect it to."

There was a previous fix to ensure `getResource` doesn't take a class name.
The convenience behavior, that `classBytes` takes either a class name or a resource
path ending in ".class", has been promoted to `ScalaClassLoader`.
For matches with two or fewer cases, @switch is ignored. This should
not happen silently.
The pattern matcher phase (conceivably, among others) can generate
code that binds local `Ident`s symbolically, rather than according
to the lexical scope. This means that a lambda can capture more than
one local of the same name.

In the enclosed test case, this ends up creating the following
tree after delambdafy

    [[syntax trees at end of                delambdafy]] // delambday-patmat-path-dep.scala
    matchEnd4({
      case <synthetic> val x1: Object = (x2: Object);
      case5(){
        if (x1.$isInstanceOf[C]())
          {
            <synthetic> val x2#19598: C = (x1.$asInstanceOf[C](): C);
            matchEnd4({
              {
                (new resume$1(x2#19598, x2#19639): runtime.AbstractFunction0)
              };
              scala.runtime.BoxedUnit.UNIT
            })
          }
        else
          case6()
      };
      ...
    })
    ...
    <synthetic> class resume$1 extends AbstractFunction0 {
      <synthetic> var x2: C = null;
      <synthetic> var x2: C = null;
      ...
    }

After this commit, the var members of `resume$1` are given fresh
names, rather than directly using the name of the captured var:

    <synthetic> var x2$3: C = null;
    <synthetic> var x2$4: C = null;
Incorporate review comments by Som Snytt.
@serialversionuid is special-cased, the warning doesn't apply.

Related to SI-7041.
Note that I removed the check to ignore @deprecated:
- @deprecated extends StaticAnnotation, so they aren't
  supposed to show up in the RuntimeInvisibleAnnotation
  attribute anyway, and the earlier check for "extends
  ClassfileAnnotationClass" makes this check superflous
  anyway.
- Otherwise, if @deprecated was extending
  ClassfileAnnotationClass it would seem inconsistent
  that we don't emit @deprecated, but would do so for
  @deprecatedOverriding, @deprecatedInheritance, etc.
  Anyway, due to ClassfileAnnotation not working in
  Scala, and the additional check which only allows
  Java-defined annotations, this is pretty pointless
  from every perspective.
Classic bait-and-switch: `isTupleType` dealiases, but `typeArgs` does not.
When deciding with `isTupleType`, process using `tupleComponents`.
Similar for other combos. We should really enforce this using extractors,
and only decouple when performance is actually impacted.
Inserted missing word "bounds".
Corrected the claimed outcome of the example in the section on repeated parameters. The example method sum sums up the _squares_ of the arguments.
roberthoedicke and others added 14 commits October 28, 2014 10:38
Inserted two missing instances of the word "the".
Corrected "invokeDynamic" to "applyDynamic".
Elided superfluous "a".
Corrected "no" to "not".
Changed two wrong plurals to singulars, and inserted a comma in an enumeration of alternatives before the last "or".
When buffering, we must report the ambiguity error to avoid a stack overflow.
When the error refers to erroneous types/symbols,
we don't report it directly to the user,
because there will be an underlying error that's the root cause.
Reverted the claimed result values of the example in the section on repeated parameters and changed the code of the example method instead.
This reverts commit 9276a12.

The change is not binary compatible, See discussion on SI-8899. Making
filterImpl non-private changes its call-sites (within TraversableLike)
from INVOKESTATIC to INVOKEINTERFACE. Subclasses of TraversableLike
compiled before this change don't have a mixin for filterImpl.
Test case by Jason.

RefChecks adds the lateMETHOD flag lazily in its info transformer.
This means that forcing the `sym.info` may change the value of
`sym.isMethod`.

0ccdb15 introduced a check to force the info in isModuleNotMethod,
but it turns out this leads to errors on stub symbols (SI-8907).

The responsibility to force info is transferred to callers, which
is the case for other operations on symbols, too.
The implementations of isAnonymousClass, isAnonymousFunction,
isDelambdafyFunction and isDefaultGetter check if a specific substring
(eg "$lambda") exists in the symbol's name.

SI-8900 shows an example where a class ends up with "$lambda" in its
name even though it's not a delambdafy lambda class. In this case the
conflict seems to be introduced by a macro. It is possible that the
compiler itself never introduces such names, but in any case, the
above methods should be implemented more robustly.

This commit is band-aid, it fixes one specific known issue, but there
are many calls to the mentioned methods across the compiler which
are potentially wrong.

Thanks to Jason for the test case!
When parsed from source, java annotation class symbol are lacking the
`@Retention` annotation. In mixed compilation, java annotations are
therefore emitted with visibility CLASS.

This patch conservatively uses the RUNTIME visibility in case there is
no @retention annotation.

The real solution is to fix the Java parser, logged in SI-8928.
The observed bug is probably solved (or should have a ticket), this 4-year-old thread dump does not seem to belong here.
This should happen every time we cut a new release. We don't touch
versions.properties files because that file specifies dependencies and
we don't want to depend on broken Scala 2.11.3 release.
Scala 2.11.4 release has been staged, we should start publishing snapshots
for Scala 2.11.5 now.
@folone
Copy link
Author

folone commented Oct 28, 2014

Added more commits (including bumping to 2.11.5-SNAPSHOT): http://typelevel-ci.orexio.org/job/typelevel-scala-pr/38/

@folone
Copy link
Author

folone commented Nov 17, 2014

Closing this, as #87 has all this incorporated.

@folone folone closed this Nov 17, 2014
@folone folone deleted the typesafe-scala branch November 17, 2014 08:49
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.