Runtime Access to Generic Types on Class Method

[saulshanabrook]

I would like to be able to access the type arguments of a class from its class methods.

To make this concrete, here is a generic instance:

import typing
import typing_inspect

T = typing.TypeVar("T")

class Inst(typing.Generic[T]):
    @classmethod
    def hi(cls):
        return cls

I can get its generic args:

>>> typing_inspect.get_args(Inst[int])
(int,)

But, these seem to get erased inside the class methods:

>>> typing_inspect.get_args(Inst[int].hi())
()

[saulshanabrook]

So the issue is that descriptors bound on generic classes that are parametrized are called on the base class not parameterized class:

This is a reduced version of the above:

import typing
import typing_inspect

T = typing.TypeVar("T")

class MyDescriptor:
    def __get__(self, obj, objtype):
        return objtype


class MyClass(typing.Generic[T]):
    x = MyDescriptor()

c = MyClass[int] 

print(c)
# __main__.MyClass[int]
print(c.x)
# __main__.MyClass

I guess the issue here is maybe that x is not in cs dict:

>>> c.__dict__
{'_inst': True,
 '_special': False,
 '_name': None,
 '__origin__': __main__.MyClass,
 '__args__': (int,),
 '__parameters__': (),
 '__slots__': None,
 '__module__': '__main__'}

So it will look it up on it's base, which is MyClass instead of MyClass[int].

[ilevkivskyi]

As I mentioned off-line this is not something easy to fix. We need some time to figure out possible
(realistic) options.

[saulshanabrook]

As a workaround, I was able to monkey patch the __getattr__ on the generic alias so that it checks if the user is trying to access a descriptor and if so, use itself as the class instead of the origin class:

import typing


def generic_getattr(self, attr):
    """
    Allows classmethods to get generic types
    by checking if we are getting a descriptor type
    and if we are, we pass in the generic type as the class
    instead of the origin type.

    Modified from
    https://github.com/python/cpython/blob/aa73841a8fdded4a462d045d1eb03899cbeecd65/Lib/typing.py#L694-L699
    """

    if "__origin__" in self.__dict__ and not typing._is_dunder(attr):  # type: ignore
        # If the attribute is a descriptor, pass in the generic class
        property = self.__origin__.__getattribute__(self.__origin__, attr)
        if hasattr(property, "__get__"):
            return property.__get__(None, self)
        # Otherwise, just resolve it normally
        return getattr(self.__origin__, attr)
    raise AttributeError(attr)

typing._GenericAlias.__getattr__ = generic_getattr  # type: ignore

With this change, both of the examples above work as expected.

[JukkaL]

In my opinion this is not something we should support even if it was technically possible. Type annotations are primarily for static checking, not for runtime purposes. Any runtime uses of types beyond very simple ones will likely hit all sorts of limitations soon enough.

[saulshanabrook]

Type annotations are primarily for static checking, not for runtime purposes.

Where is the right venue to have a conversation about supporting different runtime purposes for type hints? Would drafting a PEP to articulate a possible runtime API for using typing annotations be helpful? Or should I start a discussion on python-ideas or on discourse?

For my use case, in metadsl, I need to be able to compute the return type of a function given some arguments. I am able to do this currently using the existing runtime hooks, but it would be better if I knew I was building on solid APIs for this. I would be happy to articulate as well why analyzing the type hints led to a simpler UX in this library, I chatted with @msullivan a bit at PyCon about my use case and its relationship to mypyc (mine is at runtime to generic backends, where as mypyc is AOT to C).

[JukkaL]

@saulshanabrook The typing-sig@ mailing list a focused on discussing improvements to Python static typing. However, I suspect that most of the existing subscribers are primarily interested in static approaches. Note that mypyc also arguably mostly uses types statically (that is, during compilation). At runtime the types are erased to a quite simple form, much simpler than full PEP 484 types.

As with many other ideas, if you can demonstrate that your approach is popular among Python developers, it will be an easier sell. There are a lot of possible improvements to Python static typing and I believe that it's much easier get ideas accepted when the practical benefit relative to the complexity of the change is clear and easy to justify.

[gvanrossum]

I think the starting point should be a discussion on python-dev.

On Mon, May 20, 2019 at 07:48 Saul Shanabrook ***@***.***> wrote: Type annotations are primarily for static checking, not for runtime purposes. Where is the right venue to have a conversation about supporting different runtime purposes for type hints? Would drafting a PEP to articulate a possible runtime API for using typing annotations be helpful? Or should I start a discussion on python-ideas or on discourse? For my use case, in metadsl <https://github.com/Quansight-Labs/metadsl>, I need to be able to compute the return type of a function given some arguments. I am able to do this currently using the existing runtime hooks, but it would be better if I knew I was building on solid APIs for this. I would be happy to articulate as well why analyzing the type hints led to a simpler UX in this library, I chatted with @msullivan <https://github.com/msullivan> a bit at PyCon about my use case and its relationship to mypyc (mine is at runtime to generic backends, where as mypyc is AOT to C). — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#629?email_source=notifications&email_token=AAWCWMXKSKC6X42ESIGXMFDPWK2ZNA5CNFSM4HKZC4EKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVZCEWY#issuecomment-494019163>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWCWMTXQJU7OBG4DYEIMLTPWK2ZNANCNFSM4HKZC4EA> .

-- --Guido (mobile)

[ilevkivskyi]

@saulshanabrook One of the problems that I see (apart from maintenance burned) is that every way of supporting this I can imagine causes some performance penalty for people who will not use this (and this module is used by a lot of people). Have you tried to perform any benchmarks for various operations with the patched __getattr__ that you propose?

[ilevkivskyi]

#616 is another example where this may be useful.

[dudil]

In my opinion this is not something we should support even if it was technically possible. Type annotations are primarily for static checking, not for runtime purposes. Any runtime uses of types beyond very simple ones will likely hit all sorts of limitations soon enough.

Hi @JukkaL - just a comment as a user. Typing is very useful to anyone who is creating extensive projects in Python. It allows better readability of abstract class and code reuse (when used right).
I guess these are among the reasons why they were added in the first place.
[Sorry for not going to any debate if python is the platform of choice for extensive projects - don't think this is relevant]
Now - in the case of generic, the main issue is that it does not support any public execution time interface for getting its instance type or inheritance while the same feature is supported for regular objects. (via issubclass and isinstance). This put typing class in a very much inferior place vs. regular objects.
Yes - I'm aware I could user origin and args but they are very much limited and not common for use.
Anyway - just a user comment here for your thoughts.