Does mypy recognize classes' generic types for 3.12 syntax?

[mtsokol]

When running (latest, for 3.12) mypy check on the example below only the second instance raises the error bb = BB("bb"), which suggests that the new syntax isn't supported, is it correct?

from typing import TypeVar, Generic

T = TypeVar("T", int, bytes)

class AA[T]:
    def __init__(self, arg: T):
        self.arg: T = arg


class BB(Generic[T]):
    def __init__(self, arg: T):
        self.arg: T = arg

if __name__ == "__main__":

    aa = AA("aa")
    bb = BB("bb")

[RBerga06]

I think the behaviour you're witnessing is correct, because the PEP 695 syntax introduces a new annotation scope.

In other words, T in class BB(Generic[T]): ... (with the old syntax) is indeed a reference to T = TypeVar("T", int, bytes), but the T inside the body of AA is a different symbol.

The class AA[T]: declaration defines a new TypeVar that, because it's also named T, shadows the one defined at module scope - and because this new T does not have the (int, bytes) bound, the instantiation AA("aa") type checks.

You can think of your example as if it was roughly equivalent to the following:

from typing import TypeVar, Generic

T = TypeVar("T", int, bytes)


def _aa():
    T = TypeVar("T")

    class AA(Generic[T]):
        def __init__(self, arg: T):
            self.arg: T = arg

    return AA


AA = _aa()
del _aa


class BB(Generic[T]):
    def __init__(self, arg: T):
        self.arg: T = arg


if __name__ == "__main__":
    aa = AA("aa")
    bb = BB("bb")

I think the PEP 695 syntax you're looking for is this:

class AA[T: (int, bytes)]:
    def __init__(self, arg: T):
        self.arg: T = arg

if __name__ == "__main__":
    aa = AA("aa")

and as you can see in the mypy playground, mypy correctly reports the error now.

Note

This also means that if you are currently sharing a single TypeVar between different classes (maybe because it needs a lot of bounds), you won't be able to use it anymore to parametrize classes defined with the new syntax.
In this case, you'll have to either switch to the new syntax, at the cost of duplicating the TypeVar bounds, or stick with the old one.

[mtsokol]

@RBerga06 Thank you for an explanation! Realizing that T in the new syntax is a completely different variable and shadows the TypeVar makes sense to it all now.

I think the docs of the spec could be improved though, because the fact that for the new syntax T comes from a new scope isn't mentioned at all: https://typing.python.org/en/latest/spec/generics.html#user-defined-generic-types

It says:

There are several ways to define a user-defined class as generic:

• Include a Generic base class.
• Use the new generic class syntax in Python 3.12 and higher.

Example using Generic:
...
Or, using the new generic class syntax:

class LoggedVar[T]:
   # methods as in previous example

IMO "syntax" suggests the difference is only in a way we write it, not the mechanism it's interpreted.

This implicitly adds Generic[T] as a base class, and type checkers should treat the two definitions of LoggedVar largely equivalently (except for variance, see below).

I think there should be a note that it also implicitly defines T, and not only adds Generic[T] (and that a new scope is introduced as well). With my understanding right now it suggests that T is reused.

[RBerga06]

Yes, I agree: the spec is not very clear and might be misleading.

Also note that it's like if a single traditional TypeVar object had two, different “scopes”:

1. the scope where the symbol T exists and can be used in the source code;
2. the scope where the symbol T has a meaning as a type variable.

I'll paste here the example included in this section of the spec:

from typing import TypeVar, Generic

T = TypeVar('T')

def fun_1(x: T) -> T: ...  # T here
def fun_2(x: T) -> T: ...  # and here could be different

fun_1(1)                   # This is OK, T is inferred to be int
fun_2('a')                 # This is also OK, now T is str

As you can see, the types that T refers to in a single function call (argument type and return type) are the same - that's the whole point of a TypeVar.

However, the types that T refers to in the two different function calls are not the same. At the end of the day, it's very much like if we were calling the same function twice, with different arguments:

from typing import TypeVar, Generic

T = TypeVar('T')

def fun(x: T) -> T: ...
fun(1)                   # This is OK, T is inferred to be int
fun('a')                 # This is also OK, now T is str

In this sense, T's scope (as intended by the spec) is the scope where T has a meaning to type checkers, in this case the function signature (and body).

If you try and use T outside a valid scope, you should get an error:

from typing import TypeVar, Generic

T = TypeVar('T')
x: T = 42			# Error

and it's good, because this usage wouldn't make any sense.

However, because traditional TypeVars are runtime objects, they also have an ordinary scope, in the sense of a Python variable. This means that they can carry information (such as bounds, variance or defaults) between different, indipendent usages:

from typing import TypeVar, Generic

def foo():
	T = TypeVar('T', bound=int)
	
    def fun_1(x: T) -> T: ...  # T here
    def fun_2(x: T) -> T: ...  # and here could be different

    fun_1(42)	# ok
    fun_2("a")	# error

def fun_3(x: T) -> T: ...  # error: T is not defined

What's nice about PEP 695 is that the new syntax forces you to always define the TypeVar in the typing-meaningful scope where you'll use it, essentially making the two notions of “scope” coincide.

In fact, I believe that removing this ambiguity was one of the PEP's goals.

In my opinion, it would be better if the spec always used PEP 695 syntax whenever possible; when describing the old syntax, the docs should also highlight this important (but non-trivial) difference.