-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Add augment clauses #4043
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add augment clauses #4043
Conversation
The idempotency check is failing on First run
Second run
|
Exciting! One nice effect of object Enrichments {
implicit class RichInt(x: Int) { def increment = x + 1 }
implicit class UnwantedStringEnrichments(x: String) { def increment = x + "1" }
}
import Enrichments.RichInt
@ 1.increment
res0: Int = 2
@ "1".increment
cmd52.sc:1: value increment is not a member of String
val res52 = "1".increment
^
Compilation Failed Would augment classes have something similar? |
fa20803
to
bfab87b
Compare
Is it possible to express the equivalent of the following with an augment? class Rich[T](val self: Either[T, T]) { def leftOrRight: T = ... } I can't see how that would be expressed in the type pattern syntax without having to resort to an evidence parameter. |
for `(x, y)`: | ||
|
||
```scala | ||
augment (type T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why parens instead of brackets?
augment [type T] {
def ~ [U](that: U) = (this, that)
}
augment [type F[type A]](implicit functor: Functor[F]) {
def map[B](f: A => B): F[B] = functor.map(this, f)
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's just precedence - the type pattern syntax requires a sub-production of types. So to get a top-level type
you need to parents. By contrast [...]
would be something new and magic.
Inside an extension method, the name `this` stands for the receiver on which the | ||
method is applied when it is invoked. E.g. in the application of `circle.circumference`, | ||
the `this` in the body of `circumference` refers to `circle`. Unlike for regular methods, | ||
an explicit `this` is mandatory to refer to members of the receiver. So the following |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unlike for regular methods, an explicit
this
is mandatory to refer to members of the receiver.
Why? I can imagine implementation-dependent reasons, but it's harder to tell why it makes sense for the user.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whilst I understand the reasoning, it is also yet another local rule.
def unwrap[T](x: WrappedResult[T]): T = x | ||
} | ||
|
||
def result[T](implicit er: WrappedResult[T]): T = WrappedResult.unwrap(er) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be sure... this is a candidate for becoming def result[T]: WrappedResult[T] => T
, right?
Isn't it a bit premature to jump to the implementation before reviewing the design further? There are many interesting ideas, but I'm not sure all of them pull their weight. Right now, this use of value classes:
seems to become, after the replacement, the following:
Or did I miss something from the proposals? I haven't found a similar example, and this seems pretty much a worst-case overhead — but it's the simplest use-case of value classes. (The example is based on one from Reddit FWIW). That seems a significant amount of boilerplate, even after adding yet another way to handle extension methods to reduce the boilerplate. Modifying value classes semantics insteadWhat about modifying the semantics of value classes to forbid user-defined Controlling visibility of construction/destructionWith opaque types, you write boilerplate to decide whether to allow construction or destruction to users. But value classes allow controlling this in the standard way! class Person(private val name: String) extends AnyVal
class Person private(val name: String) extends AnyVal If some people prefer to also have opaque types to have more control and can exhibit usecases, that's also an option, but it seems implicit value classes make augmentation sugar unnecessary. A downside would be a more complicated translation, but it seems still feasible. And by our standards, we'd want to implement the translation in Also, the ones I propose seem the semantics I (and many) hoped from value classes in the beginning. I don't expect much of the implementation of value classes could be reused, but user code would be. Maybe the JVM might allow reenabling them later without overhead, which is pretty easy to transition to. It's annoying for current users of AugmentationAugmentation doesn't seem to improve much on implicit value classes. One downside with the latter is that they have names that can be used explicitly both for conversions and for types, but if it's desirable to make the conversion names optional (which I'm not sure about), it's easier to allow If explicit usage of the type name is an anti-pattern, it might be easier to deprecate that aspect of implicit classes without adding a similar feature. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's also a few assorted comments on the design documents.
} | ||
object Test { | ||
import PostConditions._ | ||
val s = List(1, 2, 3).sum.ensuring(result == 6) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, so this example introduces a magic method result
to avoid writing ensuring(_ == 6)
? I see this is allowed, but this code seems to be pretty complicated relative to a pretty modest benefit — if this were the best use-case, I'd vote for rejecting the feature. I suppose there are better ones, but I'm not sure the feature is worth its weight there.
One downside with _ == 6
is that functions are harder to optimize than implicit functions, but shouldn't inline def ensuring(inline f: T => Boolean)
work as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe people who do contracts or something like Stainless or Liquid Haskell/Scala would disagree - this is essential for them, as result
is essentially a keyword in these situations. An underscore does not cut it. Anyway, it's meant as a demonstration.
**Explanations**: We use an implicit function type `implicit WrappedResult[T] => Boolean` | ||
as the type of the condition of `ensuring`. An argument condition to `ensuring` such as | ||
`(result == 6)` will therefore have an implicit value of type `WrappedResult[T]` in scope | ||
to pass along to the `result` method. `WrappedResult` is a fresh type, to make sure that we do not get unwanted implicits in scope (this is good practice in all cases where implicit parameters are involved). Since `WrappedResult` is an opaque type alias, its values need not be boxed, and since `ensuring` is added as an extension method, its argument does not need boxing either. Hence, the implementation of `ensuring` is as about as efficient as the best possible code one could write by hand: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you do have an implementation (even though that seems premature), shouldn't this be tested?
And wouldn't you need to sprinkle quite a few inline
for this efficiency claim to be true?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be cool to add tests or benchmarks!
(elimTypeDefs.transform(tree), bindingsBuf.toList) | ||
} | ||
|
||
/** augment [<id> @] <type-pattern> <params> extends <parents> { <body>} } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is still using brackets unlike the proposed docs which use parens.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, the brackets are EBNF syntax brackets here. Feel free to change to a better notation.
|
||
### Scope of Augment Clauses | ||
|
||
Augment clauses can appear anywhere in a program; there is no need to co-define them with the types they augment. Extension methods are available whereever their defining augment clause is in scope. Augment clauses can be inherited or wildcard-imported like normal definitions. This is usually sufficient to control their visibility. If more control is desired, one can also attach a name to an augment clause, like this: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is "inherited augment clauses" a useful pattern to recommend? Maybe yes, but it seems to warrant more thought.
This is usually sufficient to control their visibility.
That seems to be a bold claim to make without spelling out the evidence. Without a case study, there seems to be a risk that neither case gives a maintainable result for typical use cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess some evidence is that a lot of other languages (Rust, C#, Kotlin) have extension methods or full augment clauses (in the case of Rust) without feeling the need for more precise scope control.
implicit class <id> <tparams> ($this: <T>) <params> extends <parents> { <body'> } | ||
|
||
As before, `<body'>` results from `<body>` by replacing any occurrence of `this` with `$this`. However, all | ||
parameters in <params> now stay on the class definition, instead of beging distributed to all members in `<body>`. This is necessary in general, since `<body>` might contain value definitions or other statements that cannot be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
beging -> being
```scala | ||
implicit class circleOps($this: Circle) extends HasArea { | ||
def area = $this.radius * $this.radius * math.Pi | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This requires two more identifiers than the augmentation above, and that seems a small benefit relative to the additional complexity being introduced, no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I disagree. It's not just the identifiers.
augment Circle extends HasArea { ... }
makes it super clear what we want to achieve. The implicit class syntax hides it to a large degree and is not intelligible at al to someone not used to it. There are 6 tokens before we even get to the thing that gets augmented, and then it's in a parameter list so visually disconnected from the thing it extends.
|
||
Conversely, it is conceivable (and desirable) to replace most usages of implicit classes and value classes by augmentations and [opaque types](../opaques.html). We plan to [drop](../dropped/implicit-value-classes.html) | ||
these constructs in future versions of the language. Once that is achieved, the translations described | ||
below can be simply composed with the existing translations of implicit and value classes into the core language. It is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like one would want to translate augmentation without using the current value class translation, no? Viceversa, if we can simplify the translation of value classes by omitting features, why not just do that?
} | ||
``` | ||
|
||
This augemntation makes `Circle` implement the `HasArea` trait. Specifically, it defines an implicit subclass of `HasArea` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
augemntation -> augmentation
```scala | ||
package shapeOps | ||
|
||
augment circleOps @ Circle { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we have top level augment clauses or they share the same restriction as implicit classes? If we can, we are missing a test case for it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently these are not possible.
title: "Method Augmentations" | ||
--- | ||
|
||
Method augmentations are a way to define _extension methods_. Here is a simple one: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about "Method augmentation is" ?
My 2 cents: the I think it may be better to introduce the concept of "augment companion" which would behave similar to how a companion does with the following exceptions:
This is not a suggestion that rethinks how extension methods are proposed in this PR, I agree. But, at least, I find it easier to explain this way. IMO, reusing existing concepts to represent extension methods is important to teach Scala developers Dotty better, as the gap between Scala and Dotty enlarges. As an aside, I echo some of @Blaisorblade's comments, but I'm not as worried about the verbosity of the new approach to define value classes in Dotty compared to the previous one. That verbosity has more to do with opaque types (and the concept of opaque companions) than the introduction of |
I'm not sure if it is a good principle that a language construct that can be used in another module or play with scoping should always be named. It helps programmers to (1) communicate & reason about the design; (2) potentially better error messages, e.g., in the case of ambiguity. Naming is also a justification of the design. If the language always requires a name for augment, the responsibility of abuse like the following is on programmer's part. If it happens locally, it's not a problem. But for it to be imported in other files, mandatory names seem to be better. augment List[Int] {
def foo: Int
}
augment List[Int] {
def bar: Int
} Can we restrict that nameless augment can only used locally, otherwise it has to be named? |
I agree that's mostly a comment on opaque types. It seemed augmentation was in part an answer to the verbosity of opaque types, but it does not reduce that significantly. On opaque types, I like the semantics, but I'm torn on the syntax. I think I understand what you hint at, but I suspect people in fact dislike the semantics of value classes. But that discussion is indeed best done in another venue. |
|
||
### Scope of Augment Clauses | ||
|
||
Augment clauses can appear anywhere in a program; there is no need to co-define them with the types they augment. Extension methods are available whereever their defining augment clause is in scope. Augment clauses can be inherited or wildcard-imported like normal definitions. This is usually sufficient to control their visibility. If more control is desired, one can also attach a name to an augment clause, like this: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/whereever/wherever/
I have a bunch of high-level comments on this feature. TBH I think it hasn't been enough thought through. The name Why call this feature "augmentation" when it is well-known in the object-oriented world as extension methods? Bringing in the term "augment" is only asking for trouble in teaching and adoption of the language. Optional names Anonymous augmentations should be very rare, not the common case. There are at least two reasons for this, some of them have been mentioned already:
These two reasons are related to one fundamental problem: this feature introduces anonymous members, a concept that has no precedent in the language. The only other anonymous thing that can come into scopes are implicit evidences for context bounds, but those are part of the lexical scope, never are they members. Based on this, anonymous augmentations should only be valid in local-to-block scopes. But unlike implicit evidences, the use cases for local-to-block augmentations should be very rare. This means that we shouldn't have anonymous augmentations at all: their existence is an additional tax on the specification and documentation that does not pull its weight. It also means that the syntax for named augmentations should be nicer, and I believe that the naming convention should be to start with an uppercase (like Augmentations with Those are misleading, as they suggest that one can make a class extend a trait or another class from the outside. However, this is not what happens, since there will be a wrapper when an instance is converted. This means that identity is broken, plus there is non-obvious boxing happening behind the scenes. The latter issue is known to have destroyed Augmentations with extends clauses should be a relatively rare use case compared to extension methods. They do not deserve special syntax. For the rare cases where the user wants this to happen, they can write their own Therefore, I think they should not be supported. Syntax TBH, I think the proposed syntax is pretty ugly, especially for named augmentations (which, as argued before, should be the default/the only ones). Together with the problem of how the feature is named itself, I would suggest something based on the key word extension rather than augment, and which gives first-class aspect to named ones. Moreover, since augmentations with extends shouldn't exist, they don't need to be catered for. Maybe something like: extension FooOps for Foo[String] {
...
} This also provides a natural generalization to generic extensions that does not need to introduce a new type-matching construct: extension FooOps[T] for Foo[T] {
} This also works for fully generic extensions: extension ToArrowAssoc[T] for T {
def ->[U](that: U): (T, U) = (this, that)
} We should also allow extensions to have implicit evidences that cannot be sugared into context bounds. Again, this syntax generalizes to that: extension FooOps[T, U](implicit ev: Babar[T, U]) for Foo[T, U] {
} (non-implicit term parameters must be disallowed, obviously) Specification subtlety The spec does not make it clear how augmentations relate to implicit conversions, and in particular does not specify the precedence of one over the other. In particular, what happens in this case: class Foo {
def bar: Int = 1
}
implicit def foo(x: String): Foo = new Foo
augment stringOps @ String {
def bar: Int = 2
}
println("hello".bar) // does it print 1 or 2? Also, in the body of an augmentation for a type augment stringOps @ String {
def substring(x: Int): String = "hijacked"
def foo(x: Int): String = this.substring(x)
} Does Relatedly, I believe that having to explicitly write |
There is one advantage to anonymity: it completely hides the underlying implicit conversion. I really dislike the fact that implicit classes can be abused as in the following: implicit class PreName(s: String) {
def toTermName = null
}
def foo(s: PreName) = s.toTermName
foo("test") |
@OlivierBlanvillain Forcing augmentations to be named is not at odds with preventing your code snippet from compiling. To do that, we just make it so that there is no type named |
@retronym's concern about how to translate class Rich[T](val self: Either[T, T]) { def leftOrRight: T = ... } is also supported out-of-the-box by the syntax I propose: extension Rich[T] for Either[T, T] {
def leftOrRight: T = ...
} |
On the method: I'd say such syntax fine-tunings need to make an even more compelling case and be held to higher standards, because they're highly visible to users while providing smaller benefits. That might include the SIP process/the contributors forum/... On the merits:
So maybe it should be At that point, maybe, implicit extension preName(s: String) {
def toTermName = new TermName(s)
} instead of implicit extension preName for String {
def toTermName = new TermName(this)
} The first is closer to what you have, the second lets you write On ambiguity, it'd be consistent to continue preventing hijacking, that's something good in Scala 2. To express that semantics, I suspect that implicit extension stringOps(s: String) {
def substring(x: Int): String = "hijacked"
def foo(x: Int): String = s.substring(x)
} is maybe (a bit!) clearer than implicit extension stringOps for String {
def substring(x: Int): String = "hijacked"
def foo(x: Int): String = this.substring(x)
} |
implicit extension preName(s: String) { is pretty good too. We could also allow implicit extension preName(this: String) { (with |
Minor nitpick. If you have
then that would imply you can also have a non-implicit
I don't see the benefit of that. Or if you can't have a non-implicit variant then (Unless perhaps if |
@Jasper-M To clarify: |
In addition, that syntax allows to make Exception to the above: |
Two quick thoughts.
|
@srjd This is a trial balloon, intended to spark discussion. It seems that's what it has achieved!
I experimented with variations of
I disagree. Note that none of the other languages that have extension methods or full augmentations (like [EDIT: I was not aware that C# does name extensions, however Kotlin and Rust don't] If we feel that optional names are not the way to go forward, I would propose to drop them completely rather than making them mandatory. If nevertheless more namespace control is desired one could always do this:
and then Regarding binary compatibility, the naming scheme is chosen so that it is controllable, I think. It means that if you have in a compilation unit several augments of the same class for the same trait, or several extension method augments for the same class, you need to add the new ones at the end. |
One other thought: if augments can only do extension methods, we still need implicit classes or have to expose implicit conversions directly. I believe the use case that a class should be made to implement a trait after the class is defined is still very important. True, we get around this by systematically using type classes, at the price of more type parameters and a lot of implicit definitions. I would like to aim for a more direct way to do this. We got used to implicit classes by now, but they are far from ideal syntax - they are are the same time too convoluted and too dense for a newcomer. Augment clauses make the intent much clearer. |
Another note on syntax:
You add something to a conceptually pre-existing package. Similarly
adds operations to |
Convert many existing occurrences of `t` as a type pattern variable to `type t`.
Some source of nondeterminism remains in the backend.
Also, display anonymous augmentation names in nicer form.
This was fixed in one of the previous commits that affeted names
rebased |
Looking for syntax alternatives, here are two that I found most appealing so far. They both keep the structure of the previous syntax but change the keywords. So far, two typical cases are:
Alternative 1: Replace
Alternative 2: Replace
What do people think? I believe |
About binary names: your suggestion addresses the objective issues I have. Subjectively I still prefer explicit names, but I won't fight for it. About your alternative syntaxes: I'm still convinced that augmentations that extend some other class/trait shouldn't exist, so ... I don't understand the quantitative "issue" you have with |
One possibility would be to allow only implementations of traits but not classes. That's sort of suggested by |
An augmentation that |
I don't think boxing behavior needs to be surprising, since the rules are clear: An extension boxes if it contains an |
Maybe trait HasArea {
def area: Int
def radius = area / 2
}
augment Cirle extends HasArea {
def area = ...
}
val c: Cirle = ..
println(c.radius) But reject the following: trait HasArea {
def area: Int
}
augment Cirle extends HasArea {
def area = ...
}
def foo(h: HasArea) = ..
val c: Cirle = ..
foo(c) // error: type mismatch;
// found : Cirle
// required: HasArea They are already two ways to achieve what this last snippet is trying to do: implicit conversions and type classes, why would we want to add a 3rd option? Also, shouldn't this be out of the scope of extension methods? |
Implicit conversions will be severely discouraged if not eliminated altogether. We should not propose them as a solution for anything. As to type classes, can you show what a solution would look like for Circle and HasArea? My current understanding is it would force you to do a global program tranformation by adding type parameters where there were none before. Or did you have something else in mind? |
Deprecating implicit conversions, then allowing augmentations with |
This would be the type class way: trait HasArea[T] {
def area: Int
}
implicit val c = new HasArea[Cirle] {
def area = ...
}
def foo[H: HasArea](h: H) = ..
val c: Cirle = ..
foo(c) I don't think we gain anything if we discourage / remove implicit conversions and simply repeat the same mistakes with Like implicit conversions, trait HasSize { def size }
trait HasArea { def area }
class Circle
augment HasArea extends HasSize { ... }
augment Circle extends HasArea { ... }
def foo(s: HasSize)
foo(new Circle) // Does not compile :-/ class Circle
trait HasArea { def area }
augment Circle extends HasArea { ... }
trait HasShape { def shape }
augment Circle extends HasShape { ... }
def foo(s: HasArea & HasShape) // How do I do that?
foo(new Circle) |
Replaced by #4085 |
Does this allow extension of objects, like (silly example) implicit class VectorHasRandom(x: Vector.type) {
def random(n: Int): Vector[Double] = Vector.fill(n)(math.random)
}
Vector.random(10) That's possible with implicit classes now, and so |
Introduce
augment
clauses. This PR is based on #4028.The first new commit 945d2a8 introduces augment clauses with a syntax that makes it easy to augment generic classes, but less intuitive to augment specific instances of such classes.
The second new commit 5060aba introduces explicit
type X
binders in type patterns, which would make it possible to base theaugment
syntax on such patterns instead.