What to do with array_like variables

[johanvergeer]

I'm currently working on core.numeric and I'm running into an interesting issue.

A lot of methods have an array_like input parameter and an ndarray as the output parameter.

For now I created a TypeVar like the following:

_ArrayLike = TypeVar("_ArrayLike", str, int, float, bool, object, ByteString, Iterable, Container, ndarray)

As an example I will use the zeros_like function

def zeros_like(
        a: _ArrayLike,
        dtype: Optional[dtype] = None,
        order: str = 'K',
        subok: bool = True,
        shape: Optional[Union[int, Sequence[int]]] = None) -> ndarray[T]: ...

This function returns an instance of ndarray containing the same type as a, or it can be overridden by dtype.

This brought me to the following conclusions (please correct me if I'm wrong)

1. a can be either
   • _ArrayLike when a is a scalar or object
   • or _ArrayLike[T] when a is a collection
2. The return value type can be
   • ndarray[_ArrayLike] when a is a scalar or object and dtype is None
   • ndarray[T] when a is _ArrayLike[T] and dtype is None
   • ndarray[dtype] when dtype is not None

I'm wondering what the type annotations of a and the return value should look like.

[shoyer]

I don't think we've turned ndarray in a generic yet, which means the return value should probably just be ndarray for now?

[rgommers]

Would it make sense to type array_like as a union of ndarray, sequence, scalar, and any/arraylike, rather than _ArrayLike = TypeVar("_ArrayLike")?

[shoyer]

Would it make sense to type array_like as a union of ndarray, sequence, scalar, and any/arraylike, rather than _ArrayLike = TypeVar("_ArrayLike")?

That seems like a reasonable start, though given the currently state of Numpy's aggressive coercion, arguably Any would be most correct. On the other hand, that would also nearly be useless for type checking, so we probably need to make some practical trade-off.

[person142]

Looping back around to this, in a non-generic ndarray world, I think Ralf's suggestion of

ndarray, sequence, scalar

(I took out any) is the right practical trade-off. I think that outside that world we get into the territory of "things that might surprise people", e.g.

>>> np.array((i for i in range(10))

When we get around to "generic ndarray" then the original question of @johanvergeer becomes wide open again; it might be a good situation for protocols (possible post #63) + a MyPy plugin. (The plugin can handle more complex things like "choose between the T of ArrayLike[T] and whatever was passed as a dtype.)

But, from the discussion in #48 with @seberg, it's looking very much like the right move re generic ndarray is to wait for NEP 41 to settle. In that case it seems reasonable to go with the "ndarray, sequence, scalar" solution for now, mostly because I suspect we should make some kind of initial release before adopting features that depend on the very latest NumPy (thus locking a lot of people out of the types for a while).

[person142]

Concrete proposal here: #66.