Allow EfficientLengthIterable equivalent for non-SDK libraries

[davidmorgan]

See #23328 for context; the SDK uses EfficientLengthIterable to decide whether an Iterable is backed by something that can quickly return "length". Because non-SDK libraries cannot access EfficientLengthIterable they are always treated as not having efficient "length", leading to extra work.

It's not clear what could be done to fix this, but if there's any way to do it with 2.0, that would be great.

[lrhn]

There really is not a good way to introduce this feature.

What we do not want is anyone, ever, writing myFunction(EfficientLengthIterable elements) .... That would be introducing a new concept that users need to worry about - it locks out all the perfectly good normal iterables and complicates things for everybody.
There is no way to make a marker interface that can't also be used as a type.

So, the answer for #23328 stands for now - use List, Set or Queue for your efficient-length needs.
It really was originally introduced to make is List || is Set || is Queue shorter for ourselves to write :)
The standard Iterable.map function preserves efficient-length-ness of the source too.

Hmm, perhaps we could expose an EfficientLengthIterableWrapper that wraps any iterable and claims to have an efficient length. It would not work as a type because List doesn't have that type, but both would have the internal EfficientLengthIterable as superclass.
It's still not pretty, and I'd prefer not doing it.

We'll think about it, but it's not high on the list of 2.0 features.

[davidmorgan]

Hmm. Actually I think I have a workaround.

I don't want to implement List/Set/Queue because they carry methods that I won't implement. (The Java/Guava route for immutable collections). But I can do this:

class BuiltList<T> implements Iterable<T> {
  factory BuiltList() => new BuiltListImpl<T>();
}

class BuiltListImpl<T> implements BuiltList<T>, List<T> {
}

That means I get the static checks I want. At runtime, instances will be EfficientLengthIterable via List.

This does mean that if anyone is doing processing based on the runtime type and checking for List they may be surprised ... but I can't see that being particularly important.

WDYT? Do you see any downsides to doing this? Thanks!

[davidmorgan]

My workaround is slightly less nice than hoped :/ ... because BuiltList methods clash with List, and BuiltSet with Set.

But I can use/abuse Queue, since none of its methods clash with mine and they're just going to throw UnimplementedError anyway.

Benchmarking List#addAll(new BuiltList(...lots of data)) shows a 2x speedup, as you would expect -- it can straight away grow the list by the right amount.

@lrhn it's not pretty but I think I can live with this. What do you think, please?

[lrhn]

@floitschG

I think it's an extremely ugly hack, but I can also see that we are not giving you a better alternative.

I'll consider whether to expose a subclass of EfficientlengthIterable in some way, so you can implement that, but it can't be used to detect/require iterables with efficient length (since List doesn't implement that particular subtype). It's still ugly, and it's sort-of sad that users can't check if an iterable has efficient length and use it themselves.

Maybe (and definitely no promises):

class EfficientLengthIterable implements _internal.EfficientLengthIterable {
   factory EfficientLengthIterable._cannotBeExtendedOrInstantiated() => null;
   static bool isEfficient(Iterable iterable) => iterable is _internal.EfficientLengthIterable;
}

That would allow class MyIterable implements EfficientLengthIterable { ... }, but not meBeingClever(EfficientLengthIterable iter) => ... (which we want to prevent, and evidence shows that it will happen if we don't prevent it). The latter won't work because it rejects List, and nobody wants that :)
We will still get code like:

meBeingTooCleverByHalf(Iterable iter) {
  assert(EfficientLengthIterable.isEfficient(iter));
  ...
}

but we can't prevent all self inflicted wounds like that, just recommend that they say List instead.

[davidmorgan]

What about splitting EfficientLength from Iterable altogether -- then making it public?

abstract class EfficientLength {
  int get length;
}

List would then implement both EfficientLength and Iterable.

There's then no point in writing

void myFunction(EfficientLength e)

because you can't do anything useful with it.

[floitschG]

That's what we had before. (Not publicly exposed, but still).

We merged them, because least-upper-bound computations found EfficientLength on things like [ new List(1), new Set()].

Already at that time, we debated, if we should expose EfficientLength or not... It just feels wrong to have such a class in the core libraries.

We are now wondering, if we can reduce the need for it. Most of the time, when we use it, is because the API accepts an Iterable. In many cases, we should just request a List instead.

[davidmorgan]

There probably aren't too many important uses, but List#addAll seems to be one.

[lrhn]

When we had EfficientLength by itself, it wasn't clear what it meant. There was at least one case of it being used on a Map implementation, which it was never intended for (all maps should have efficent lengths, and nobody ever checks it on anything but iterables anyway). That, and the odd least-upper-bound behavior, was why it became EfficientLengthIterable.

[pschiffmann]

What do you think about using an annotation as a marker?

import 'dart:collection' show efficientLength;

@efficientLength
class BuiltList<E> implements Iterable<E> { ... }

[davidmorgan]

@pschiffmann I don't think an annotation helps -- there's no way to check for it given an instance of that type.