Are transparent extension types in a subtype relation with their on type?

[leafpetersen]

The initial extension type proposal from @eernstg suggests that an extension type is a supertype of its on type.

Ignoring the question of whether transparent extension types by default are the right choice for now (that is, whether implicit assignment into the type is what we want), there is a question of whether doing so via subtyping is the right choice. There are at least three alternatives, given an extension E with on type O:

• Make O be a proper subtype of E
• Make O be equivalent to E (i.e. treat it as a typedef in the equational theory)
• Make O be assignable to E (that is, treat it as if there were an implicit coercion from O to E)

Examples:

extension class E on int {}
E e = 3; // Valid if `E` is a supertype of `int`, or if `E` is equivalent to `int`, or if `int` is assignable to `E`
int i = e; // Valid only if `E` is equivalent to `int`
List<E> x = <int>[]; // Valid only if `E` is a supertype of `int`

Ignoring the technical issues around handling the subtyping which I will file a separate issue for, what are the arguments for using subtyping vs assignability vs equivalence?

[lrhn]

So, the options here assume that we want assignability from O to E. Is the question really about the subtype relation between O and E, or just about how to ensure assignability?
(Having O be a proper supertype of E is possible too, but doesn't provide assignability, so we'd need implicit coercion as well, but it's a different subtype relation).

The "equivalent to" case makes the type a "typedef", so a type alias. This differs (as I read it) from being a separate type with mutual subtype relations (like the top types are). For assignability, the two are equivalent.

For assignability, I prefer equivalence. I don't really care at the base type, but if we're comparing List<E> to List<O>, I want to be able to assign freely between the two.

It means that I can write a function which internally uses an extension type, say, to create a list of O objects, add elements to it and return it as a List<O> without having to copy at any point. The subtyping relationship doesn't matter much at the base type, we can do whatever we want, but when I have a List<E> and want to return a List<O>, making E a proper supertype of O forces me to copy the list to change the type argument.

I see (unprotected) extension types as convenience type aliases. Instead of doing E(o).foo() everywhere, I just assign o to E and it happens automatically.
So, I really do see (and want to see) the type E as a type alias with an extra static-only flag which changes the static behavior of member invocations. That supports the use-cases I am envisioning.
At run-time, it's all O. Just like dynamic is really an alias for Object? with different static member invocation rules.

If I have to copy in either direction of List<E> to/from List<O>, then I lose the zero-overhead that I want.

[eernstg]

One of the motivations for having a zero-cost abstraction mechanism that always comes up is that we can then (without a performance hit) use the same underlying representation for several different purposes, and get help from the type system to keep them separate.

Hence, I'm a little worried about type equivalence, because it allows us to assign freely between extension types, not just to/from the on-type:

// An int could represent some very different things:
extension type Age on int {...}
extension type Temperature on int {...}

void main () {
  Age a = 42;
  Temperature t = -7;
  a = t; // Oops!
}

The proposal in #1452 uses a subtyping relation in order to allow "entering" the extension type implicitly, but requires an explicit cast to "exit" the extension type again.

If we choose assignability, then it could be in any direction, or both directions, or none, so that's an orthogonal discussion. But it's assumed here that we have an implicit (possibly no-op) coercion from an instantiated on-type to the corresponding extension type.

I think the usability of the mechanism will suffer unnecessarily if this implies that the following error occurs:

extension class E on int {}
List<E> x = <int>[]; // Error.

I'd like an extension type to be usable with an object structure that hasn't been created with that particular extension type in mind.

This is of course extra critical if it is a large object structure (e.g., a big tree representing a protobuf or JSON value), and if that object structure was created by a general library that doesn't know about all the extension types we might define.

So we might receive data from a business partner as JSON, agreeing to use a specific schema. We'll use a general-purpose JSON parser to obtain a "typeless" representation of that value, but then we'll use it based on some extension types that know how to read the given tree assuming that it conforms to that schema.

I think that will only work if we allow List<T> <: List<E> for a given extension E on T {...}.

[lrhn]

Having thought about it more, I can see both points. So, I want both!

Let normal extension declarations introduce extension types (which auto-show all non-shadowed instance members of the on type) and which is equivalent to the on type in subtyping, like a tagged type alias. Erase the tag at run-time.
This allows you to freely cast values between different extensions of the underlying type, which is what they are: The underlying type ... with a few more members on top. That's also how those extensions were originally designed. Allowing them to be used as a type is not to introduce an abstraction, it's to introduce a shorthand for avoiding extension conflicts.

Then let extension type declarations introduce a type which is an abstraction over the underlying type. Make that a super-type of the on type and a subtype of Object?, allowing easy entry and requiring an explicit exit from the abstraction.
It's not encapsulation, you can extract the underlying value and modify it outside of the abstraction, you just have to do it explicitly.

Which kind of extension you want probably depends on the use-case it's made for.
If I make a StringExtension with more string functions, it's because I want to call those extra functions.

(I'm not sure protected extension types are worth it, they're too easy to break anyway).

[fishythefish]

I was originally in agreement with Erik about the issues with type equivalence, but I'm more okay with it given Lasse's suggestion that there be separate syntaxes for introducing type-equivalent and -inequivalent extensions.

In the extension type case, I'd prefer we support easy entry into the extension type by assignability rather than by O <: E. Aside from the issues brought up in #1465, this clashes with my intuition about what subtypes are and the relative width of their interfaces. In Erik's example, we'd have int <: Age and int <: Temperature, but these "is-a" relationships seem backward. (I'd be okay with E <: O, but this alone doesn't save you from having to perform the coercion.)

[lrhn]

It's true that we are using "subtype" as merely meaning "Coercible from".

For an extension type Ext on On ..., if we make On <: Ext then we're saying that every On is an Ext. That's true, because every value of type On can be used as an Ext. It's substitutable for an Ext. By not doing the other direction, we're saying that an Ext is not (just) and On. Even though every Ext value literally is an On, so that's not the underlying truth—it's the abstraction truth.
We treat Ext as a (semi-)opaque abstraction, where we hide (without actually encapsulating) the underlying representation.
Allowing you to assign On to Ext is a way to enter the abstraction. Since we allow all values into the abstraction, this is a safe operation. Not allowing you to assign Ext to On is an attempt to preserve the abstraction and hide that it's really an On.
That behavior is modelled by On <: Ext.

If we go in the other direction, Ext <: On, then we're saying that all Ext values are On values, and you can throw away the Ext-ness at any time simply by assigning to On. On the other hand, it's harder to make something an Ext, you have to explicitly cast (probably either (o as Ext) or Ext(o)). If we don't actually restrict which values can enter Ext, then that's unnecessary overkill, and we might as well just have assignability in both directions.

If we do restrict which values can enter Ext, then a subtype would work*mdash;*the values of a subtype are a subset of the values of the supertype. A completely separate type would work too.

So, the different directions of assignment means different things (depending on perspective).

If we restrict the values which can enter the extension type, then we don't want On <: Ext. We can have Ext <: On or not (subtype or unrelate type, either can work).

If we do not restrict the values which can enter the extension type, then making On <: Ext is safe, and you have easy assignment to enter the extension type. We can have Ext <: On or not (either it's a protected abstraction, or it's just a type alias with assignment in both directions).

So, four models:

• No relation: A separate type with explicit mapping in both directions. Only worth it if we can restrict which values enter the extension.
• Ext <: On, not On <: Ext: The extension type is hard to get into by accident, easy to get out of. Only worth it if we can restrict which values enter the extension.
• On <: Ext, not Ext <: On: The extension type is an abstraction hiding its implementation. Any value can enter the abstraction, but after that, you need an explicit cast to get out.
• Both directions: The extension type is a static type alias which you enter and exit at will. It's just a way to easily get a different set of operations available. A convenience.

I'm not sure I believe that the first two are worth the effort needed to actually ensure the subset property. Or that, if we want it, that it's the best way (q.v., #1467).
The last two are the ones I'm personally arguing for as static-only extension types. I can see use-cases for both.

[eernstg]

@lrhn wrote:

Let normal extension declarations introduce extension types (which
auto-show all non-shadowed instance members of the on type) and
which is equivalent to the on type in subtyping, like a tagged type alias.
Erase the tag at run-time.
...
Then let extension type declarations introduce a type which is an abstraction
over the underlying type. Make that a super-type of the on type and a subtype
of Object?, allowing easy entry and requiring an explicit exit from the abstraction.

That's certainly an interesting approach!

One issue, though: If a non-explicit extension introduces a type E (generic or not) which is subtype equivalent to the corresponding instantiated on-type, and it allows static shadowing of some instance members, then any implicit cast from the on-type to some other type (in particular, to the on-type itself or to any supertype thereof) will eliminate the extension methods that may shadow instance methods and thus silently switch implementation:

extension E on num /*show num hide floor, implicitly*/ {
  // We want this non-standard `floor` semantics, so we shadow `num.floor()`.
  int get floor() => 0;
}

void main() {
  E x = 42.2;
  print(x.floor()); // Prints '0'.

  void printFloor(num n) { print(n.floor()); }
  printFloor(x); // Seems very similar, but prints '42'.
}

Because of this, I think the completely frictionless introduction and elimination of the extension type should be reserved to the case where we do not have any "static overriding", that is: Whenever the on-type has a member m, the extension type has the same implementation of m (which could be achieved using show m, implicitly or explicitly), or the extension type does not have a member named m at all.

This would help us avoiding the tricky "static override" behavior in the case where the static type is so extremely easy to enter and leave implicitly.

[eernstg]

@fishythefish wrote:

In Erik's example, we'd have int <: Age and int <: Temperature, but
these "is-a" relationships seem backward.

I agree that this particular subtype relationship doesn't follow the normal rules associated with OO subtyping: The interface is in general replaced rather than extended.

We could argue that we should use the term 'assignable' rather than 'subtype', simply because 'assignable' doesn't seem to promise "has a superset of the members". If we can provide the same affordances then I don't care how it's specified: The outcome is in any case that certain pairs of types admit assignment/argument-passing/return etc., and other pairs don't.

However, I would very much like to allow for higher-order access to entities using extension types, without requiring those entities to be created with the expectation that they should be used in this manner. For instance:

extension E on int ... {...}

void main() {
  List<E> xs = e; // Some expression with static type `List<int>`.
  ... // Work on the elements of `xs` using the discipline encoded as `E`.  
}

So if the given notion of assignability does not extend to type arguments (and other composite types) then I really think we're going to drop a useful feature on the ground.

[eernstg]

@lrhn wrote:

So, four models: ...

This analysis is perfectly compatible with the view I've had on the situation. I do wish to mention the following, though:

• No subtype relation This fits well with some kind of protection, hence I'm using 'protected extension type' to indicate that this rule is used. This is indeed most useful if we can control which instances get to have the extension type, but the constructors and type cast/test rules of a protected extension type E provide a strong guarantee for exactly that: If e has type E then the value o of e is an object that was returned by a constructor of E. If the invariant-related state of o is final, or there is no alias to e then the invariant will be preserved. There are loopholes (say, aliases or dynamic invocations), but the support for invariant enforcement is definitely better than nothing.

• Ext <: On, not On <: Ext I tend to think that this isn't particularly useful.

• On <: Ext, not Ext <: On I prefer this as the main rule, because it keeps distinct extension types separate (so we don't silently allow assigning an Age value to a Weight variable), and it allows for convenient adoption of the added discipline.

• On <: Ext <: On If we actually want to enter and exit this kind of extension type silently, why wouldn't we just use implicit invocation? That is, why wouldn't we simply use an extension declaration (as of today, with no extra keywords) as a set of extension methods? (So we're not using extension types at all, we're just using the existing extension declarations, with the existing static analysis and semantics). Anyway, if we do wish to support this case then I think we should enforce that there is no static overriding (for instance, we can't have extension E on int ... { bool get isEven ...} such that isEven is an extension method declared by E, and also an instance member of the on-type int).

[lrhn]

• Ext <: On, not On <: Ext I tend to think that this isn't particularly useful.

It's a consistent view of a subset type. If you define a type, Choice to represent a subset of strings containing only, say, {"yes", "no", "maybe"}, then it's reasonable to treat all Choices as strings, but not all strings as Choices.

The protected extension type described here is (potentially) such a subset with an added abstraction on top. The abstraction is what removes the Ext <: On, like it does between (my version of) the extension and extension type.

For:

• On <: Ext <: On

The reason for not just using implicit invocation is that by casting to the extension type, you avoid conflicts. The members of the extension type always win, even if there are other extensions imported. It's a shorthand for saying "always use this extension", without having to explicitly write Ext(o) everywhere.

[lrhn]

I don't think Ext x = o; o.m(); should mean something different from Ext(o).m().
The real question is what they then mean.

For a plain extension, not an extension type, I lean towards allowing members of o to show through where not shadowed by extension declarations (with an option to control it, but the default would be show-all).
This is backwards compatible with the current behavior where Ext(o).oMethod() is a compile-time error (so, no currently valid program changes behavior).
It means that Ext(o).method() means the same as o.method() if there is no name conflict and no extension conflict. Using Ext(o) or Ext e = o; are ways to handle conflicts, not any attempt at introducing new abstractions.
You can assign back and forth with no restriction, it's still the same value, it's all about how you choose extend that value.

For an extension type, you don't get a default show-all, you have to explicitly write what you want to show. It is a new abstraction, and the type is significant.

[leafpetersen]

It's true that we are using "subtype" as merely meaning "Coercible from".

@lrhn I'm not sure if this is just a terminological confusion, but no we are definitely not. That's exactly the point that caused me to file this issue: there is a model in which we mean "coercible from" which allows us to assign an O to an E (because they are coercible) but not a List<O> to a List<E> because they are not subtypes; and a separate model in which we mean "supertype of" which allows us to assign an O to an E because they are subtypes, and also a List<O> to a List<E> because they are subtypes.

In principle, we could explore a model in which the "coercions" are defined compositionally over types so that you get List<O> coercible to List<E>. It's actually a somewhat interesting model to explore, but I'm not entirely sure what it buys you over the subtyping model. I guess it comes into play in override checking, and maybe the runtime semantics?