Types for itertools are generic in stubs but not at runtime

[hurryabit]

The type definitions in stdlib/itertools.pyi are misleading in the following sense:

When I write something like

def foo():
    return itertools.product("ab", [1, 2])

the local type inference of various Python type checkers suggests that foo returns something of type itertools.product[tuple[str, int]]. (This makes perfect sense given the type definitions for itertools.) However, this type cannot be used in type annotations since itertools.product does not implement __class_getitem__. I would consider this suboptimal UX.

Since the classes in the itertools module are not generic types (in the sense of implementing __class_getitem__), wouldn't it be better if the typings gave type checkers the opportunity to infer expressible types, such as Iterator[tuple[str, int]] in the example above? I can see two ways to achieve this:

1. Change the return types of the __new__ methods to return Iterator[...].
2. Pretend the classes defined in itertools are functions returning Iterator[...] rather than classes. This is how they are used.

In either case, local type inference will suggest types can actually be used in type annotations.

What do people think about these ideas? Is it worth, preparing a PR for this?

[srittau]

We try to be accurate in our return types. While it's unfortunate that itertools.product is not generic at runtime, it is at type checking time, and a generic itertools.product can be used in type annotations with either the annotations future import or by quoting it:

from __future__ import annotations

import itertools

x: itertools.product[tuple[str, int]]

[AlexWaygood]

You should be able to use Iterator[tuple[str, int]] as the return annotation for your foo function, @hurryabit. While typeshed tries to be as precise as possible in its return types and as faithful as possible to the runtime, in your own code you'll probably want to use Iterator in return annotations rather than a specific type from itertools. The itertools types are (with some exceptions) mostly indistinguishable from each other in their public interfaces, and using Iterator as the return type for foo means that your typing won't break if you switch to using a different iterator class or function in the future.

There's some related discussion on this at python/cpython#112896.