Skip to content

Should extension members take precedence over instance members? #556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
leafpetersen opened this issue Aug 31, 2019 · 11 comments
Closed

Should extension members take precedence over instance members? #556

leafpetersen opened this issue Aug 31, 2019 · 11 comments

Comments

@leafpetersen
Copy link
Member

Discussion around the interaction between sealing and extension methods re-raised the question of what the right behavior should be when an invocation a.m is performed and m is present both as an instance member in the static type of a and as an extension that otherwise applies to a.

  • If the instance member takes precedence, then adding an instance member is breaking since extensions may exist with that name that will no longer apply. This is true anyway because of extension and implementation: however, if we add sealed classes to the language, then sealed would no longer imply that adding members is non-breaking.

  • If the extension member takes precedence, then adding an extension member is breaking since instance members with that name may exist.

Summary of previous discussion.

In comments, @lrhn makes the following points:

  • There are readability benefits to choosing the instance member. If you see f.then where f has static type Future, a reader will be very misled if the .then is not actually resolving to the member declared on Future.
  • "there is no way to override the extension member if it takes precedence over the instance member"

@nex3 responds that:

  • An author of code who intentionally calls an extension method will be unpleasantly surprised if the meaning of the code silently changes because someone added a method which now overrides the extension that they were calling.
  • "extension methods always know the interface they're extending and can avoid shadowing existing names, whereas a class doesn't and cannot know the set of all libraries that define extension methods for it."

@lrhn responds

  • "If we act as if any extension in scope is always deliberately there, then it might be reasonable to consider it a stronger signal than the interface member."
  • "The question is then whether "extensions are always deliberate" is a reasonable assumption. ... I'm not entirely sure that it is a safe assumption. I expect packages to start introducing extensions, and those extensions will then be imported implicitly by anyone importing the package."

@eernstg Makes the points that:

  • The author of an extension may not be aware of subclasses of the target type, and/or future modifications of an interface may invalidate the assumptions under which an extension was written
  • If extension members take precedence, then code authors must always take extensions into account, since even apparent invocations of well-known methods on well-known classes might not actually be doing what they appear.
  • Tooling might help

@leafpetersen Re-iterated the concern about sealing:

  • If we add sealing to the language, we might need to disallow extensions on sealed classes to get the programmer benefit of sealing
  • We would still get the modular compilation benefits, however
  • We could consider adding a modifier (e.g. override) in order to allow extensions to be written that do override the instance method, and hence could be allowed to apply to sealed classes .

Other languages

Kotlin and C# both resolve this in favor of the instance member when overloading does not otherwise resolve it. Since both languages have overloading, adding an instance member is less likely to break code that uses existing extensions, but it still can break it.

Both Kotlin and C# allow extensions on sealed classes.

cc @nex3 @eernstg @munificent @lrhn @srawlins

@ds84182
Copy link

ds84182 commented Aug 31, 2019

Just my 2¢, I believe that extensions from other libraries should be opt in, while extensions in your library cannot be opted out of.

Something like:

import 'package:foo/foo.dart' use ExtA;

or

using ExtB, ExtC;

Which can declare that the library is opting into those extensions, and that they should override instance members.

For Rust traits, it is also true that it needs to be in scope. However, the import discipline is different between the two languages, where Dart prefers to import all (like ::*). Since we do not have the same import discipline, I don't think it's a good idea to throw extensions into libraries that haven't opted into them. It can make adding an extension to a library a breaking change.

@lrhn
Copy link
Member

lrhn commented Sep 3, 2019

@ds84182
You are suggesting that extensions are not imported automatically, but have to be listed explicitly in the import statement to be available.

It would have to be a new syntax, not show or hide because show hides everything else and hide is not explicit about what it includes.

That is an option. Define imports as not including extensions by default, but only if they are mentioned in a use clause. It does make some use-cases harder, like where a framework author wants a number of extensions to be available, but everybody has to list all of them by name every time they import the library. Maybe that could be shortened to use *, but it's still more than just importing the library.

@eernstg
Copy link
Member

eernstg commented Sep 3, 2019

There are two topics on the table here:

  • When an instance member as well as an extension member ('extension instance member', actually) matches an invocation, which one has the higher priority?
  • How do we enable an extension, such that an implicit extension method invocation can occur?

The priority topic is in the title of this issue. We could change it to "extension wins" or "extension member wins if declared to do so", or "an ambiguity is an error", but all of these would require new syntax for specifying at a call site that the instance method should be selected.

Orthogonally, we could reduce the potential for conflicts by improving control over the situations where each extension is enabled, which is again new syntax. We talked about constructs like import 'lib.dart' enable MyExtension;, or disable MyExtension or we could have disable extensions. We also discussed a declaration like extension E = myPrefix.E; (which could be used to enable myPrefix.E in any given scope), and we had the proposal here that extensions should never be enabled unless some explicit syntax is used to turn it on.

I'd propose that we keep override declarations ("extension member wins because it's declared to do so") in mind. It should not be hard (or breaking) to add this if the need arises.

For controlling which extensions are enabled, we can actually rely on source code organization for now: Extensions can be located in separate libraries (one by one, or as small clusters of extensions that are intended to be used together), and such libraries can be gathered into larger clusters (e.g., by having one library in each relevant package, exporting all extensions in that package). This would give developers the same degree of control as any library-level enablement mechanism.

So there's no panic on those issues.

But if we want to give extension methods a higher priority than instance methods in general, then we have to do it now, and we need the new override syntax. As you probably know, I'd prefer to keep the high priority of instance methods, primarily because it's conceptually wrong to have a model which says "an extension method is an override of every instance method of the on type with the same name" (the semantics are just wrong for that interpretation).

@ds84182
Copy link

ds84182 commented Sep 4, 2019

@lrhn For framework authors, one idea I was considering was having extensions declared in the same library as the target type be use-by-default. Otherwise, dart:ffi wouldn't work as great.

Next, for extension resolution. I would like extensions to have a higher priority over instance methods with the same name for many of the reasons described here: dart-lang/sdk#38107. Yes, a user could shoot themselves in the foot by declaring an extension that, say, changes the implementation of int.operator+ to one that subtracts instead. But that is up to the user. What you want to guarantee is that adding and using an extension on a class does not break later when that class gains a method with the same name. We have the same issue with top level members, but we also have the ability to show and hide members at a library level. Perhaps the use syntax could also allow for hiding specific methods from an extension.

Also, it would be rather nice for use to be scoped (like how Rust allows you to import traits into a block's scope). Would be rather useful for temporarily using an extension without opting the entire library into it.

mod foo {
    pub trait Bits {
        fn bits() -> u8;
    }

    impl Bits for i32 {
        fn bits() -> u8 { 32 }
    }

    impl Bits for i64 {
        fn bits() -> u8 { 64 }
    }
}

// Similar implementation but with a different type
mod bar {
    pub trait Bits {
        fn bits() -> usize;
    }

    impl Bits for i32 {
        fn bits() -> usize { 32 }
    }

    impl Bits for i64 {
        fn bits() -> usize { 64 }
    }
}

pub fn main() {
    {
        use foo::Bits;
        println!("{} {}", i32::bits(), std::mem::size_of_val(&i32::bits())); // 32 1
    }
    {
        use bar::Bits;
        println!("{} {}", i32::bits(), std::mem::size_of_val(&i32::bits())); // 32 8
    }
}

@lrhn
Copy link
Member

lrhn commented Sep 4, 2019

@ds84182

For framework authors, one idea I was considering was having extensions declared in the same library as the target type be use-by-default. Otherwise, dart:ffi wouldn't work as great.

That opens a can of worms about what it means to be declared in the same library. Most of our packages declare each class in a separate library, then exports all of them from the single main entry-point of the package.
A user of the package has no idea which private library, like lib/src/something/foo_impl.dart, actually declares the Foo class exported by package:mypackage/mypackage.dart, and just as little idea which library declared an extension on Foo. If the behavior is predicated on whether those two are from the same library or not, users do not have the tools to make reasonable predictions about how things behave.
(If we said "same package", then maybe users would be closer to be able to recognize it, but packages can export members of other packages as well, so it's still not clear-cut).

I'd be very very wary about having any semantics depend on the source location of a declaration. Dart semantics are about accessibility of declarations through imports and exports, the original declaration is something you can change in refactoring without changing behavior, and that's a good thing.

@rrousselGit
Copy link

To "Should extension members take precedence over instance members?", I would say no.

In fact, I would argue that it should be a compile error to define an extension member using the same name as something already defined in the extended type.

If an extension mistakenly overrides an instance member, this would be very difficult to track/debug.
That's something impossible to find just by looking at the source code, especially if the extension is defined in an imported file.
And while tooling may help, not everybody uses tools and these are not available in code-reviews, articles, or other mediums.

This also means that upgrading dependencies may suddenly break objects unrelated to the dependency.

@ds84182
Copy link

ds84182 commented Sep 4, 2019

@rrousselGit then you have the opposite problem for users of extensions.

If a single package updates to add a member to a class, something that typically isn't a breaking change, it now has the ability to break downstream packages if they declare an extension with the same name. This extends for all types, since you can make an extension on Object?. While the same thing can be said about implements, a user using implements on a class that is documented as "do not extend/implement" is opting into hacky behavior.

The same thing can be said about all packages in general. You're able to add a conflicting top-level member to a package, and it is up to your consumers whether it becomes an error or not.

With explicit extension use, this becomes somewhat traceable. You're protected against extensions adding new members and classes adding new members, as long as you don't use a glob import.

// optional.dart
extension Optional<T> on T? {
  X map<X>(X Function(T) f);
}

// library.dart
// Example syntax:
import 'package:optional/optional.dart' use Optional; // or Optional.*, or Optional.map

int parseAndIncrement(String a) => int.tryParse(a).map((n) => n + 1);

If it could become an error to upgrade your dependencies, I can definitely see libraries prefixing all their extension member names with their library name, to avoid conflicts:

extension Foo<T> on T {
  void fooDoBarThing();
}

Kinda unfortunate.

The next issue is that for core SDK types, it becomes a breaking change to add a member to even the core types, like int, bool, and String. It wasn't breaking previously, and should not be breaking at all. It limits the ability for dart:core to grow. Restricting extensions on core SDK types makes extensions useless for me. For example, take this bit of code:

extension ToHex on int {
  String toHexString([int width]) { ... }
}

Which somewhat implements dart-lang/sdk#6618, an issue that has been open since 2012.

Extensions should be seen as importing locally scoped methods on existing objects, and is the equivalent to calling a static method with syntactic sugar.

@ds84182
Copy link

ds84182 commented Sep 4, 2019

@lrhn What do you think about extensions declared inside the declaring class (I personally don't like it, but w/e):

class Pointer<T extends NativeType> {
  // Alternatively, "extension where T is _NativeInteger"?
  extension on Pointer<_NativeInteger> {
    int load();
    void store(int v);
  }
}

Feels extremely weird.

@rrousselGit
Copy link

rrousselGit commented Sep 4, 2019

then you have the opposite problem for users of extensions.

If a single package updates to add a member to a class, something that typically isn't a breaking change, it now has the ability to break downstream packages if they declare an extension with the same name.

That's why I added that, IMO, shadowing an exiting member with an extension method should be a compile error.

In that situation, worst-case scenario, the app will just not compile.
It's a lot better because this eliminates the possibility of shipping the app without solving the name conflict.

@leafpetersen
Copy link
Member Author

What do you think about extensions declared inside the declaring class (I personally don't like it, but w/e):

I believe that Kotlin supports local extensions (not sure about class local, but I think function local). We considered allowing function local extensions, and I actually kind of like the idea. If you think of extensions as just a way of defining overloaded static functions that get to use the . syntax as the call operator (as I do), then it makes a lot of sense to allow function locals. It allows you to have the extension close over something from the surrounding scope. In the end we decided it wasn't worth its weight though. Also, the syntax we chose is heavier than the Kotlin syntax, so defining function local extensions didn't feel quite as natural.

@leafpetersen
Copy link
Member Author

Ok, thanks for the feedback everybody. We've discussed this fairly extensively at this point, and the consensus on the team is that there are tradeoffs with either, neither design clearly dominates the other, and that on balance we feel that the current design is the correct one for our purposes. Key issues weighing in favor of this decision include (at least) the following.

  1. Our intention is that extension methods serve as a way of augmenting existing types or adding generic helper methods on types, rather than as ways of replacing the existing API on a type with a new one. For the latter purpose, we would like to eventually add a separate facility (see related issues already filed in this repo). With this intention in mind, it seems more correct to prefer the existing API on types rather than allow them to be replaced.

  2. We would also like to have the ability to add extension methods to the core SDK libraries. This is essentially not possible as a non-breaking change if we prefer extension methods over instance methods.

  3. From a readability perspective, we feel that being able to trust that a call to a known method name that exists on an instance is actually a call to that method without knowing what extensions may or may not be in scope is overall a win.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants