Enum errors with generics

[SimpleArt]

Bug report

In python < 3.9, creating a subclass of both enum.Enum and typing.Generic (or a subclass of typing.Generic) causes an error.

from enum import Enum
from typing import Generic, TypeVar

T = TypeVar("T")


class Foo(Generic[T]):
    ...


class Bar(Foo[int], Enum):
    x = 0

Error:

Traceback (most recent call last):
  File "test.py", line 11, in <module>
    class Bar(Foo[int], Enum):
  File "...\enum_newest.py", line 566, in __new__
    raise exc
  File "...\enum_newest.py", line 275, in __set_name__
    enum_member = enum_class._new_member_(enum_class, *args)
  File "...\Python38\lib\typing.py", line 875, in __new__
    obj = super().__new__(cls, *args, **kwds)
  File "...\enum_newest.py", line 1132, in __new__
    raise ve_exc
ValueError: 0 is not a valid Bar

Trying this with newer versions of enum.py does not appear to solve the issue, meaning it probably has something to do with a change in typing.Generic in python = 3.9 that happened to fix it. I wasn't able to test newer versions of typing.py since it was pretty complicated and used imports not available in previous versions.

I also couldn't find anything about such a change to typing.Generic, and I'm not sure if this is a more general issue concerning how enum.Enum plays with special situations considering the current enum.py still gives the above error.

[zware]

It's not clear to me if you're saying there's a bug in 3.9 and earlier, or 3.10 and later. If the bug is in 3.9 and earlier, it's not going to be fixed: 3.9 and before are in security-fix-only maintenance mode.

[SimpleArt]

@zware
Here's a breakdown of the situation from what I understand after playing around with it:

• Python 3.8 and earlier has the bug. Its version of typing.py has something which causes an error with enum.py.
• Python 3.9 and later probably still have the bug, but it is no longer reproducible using typing.Generic. Its version of typing.py seems to have accidentally fixed the specific case of this issue in typing.Generic, while its version of enum.py still produces the bug on the old typing.Generic. Perhaps this is not an issue with enum.py and just an issue with the old typing.py, but I could not find any mention of such.

In other words, I think the bug still exists in enum.py but is no longer reproducible with newer versions of typing.Generic. You can reproduce this by taking the newest version of enum.py back to Python 3.8, something like this:

from future_enum import Enum

[SimpleArt]

I managed to narrow the bug down to #21327 (Remove the new method of typing.Generic), which removed Generic.__new__ for performance reasons. This happens to have a conflict with:

https://docs.python.org/3/library/enum.html#supported-dunder-names

__new__(), if specified, must create and return the enum members; it is also a very good idea to set the member’s _value_ appropriately. Once all the members are created it is no longer used.

It just happens to be a complete coincidence that this affects how typing.Generic combines with enum.Enum.

Looking through typing.py suggests this was the only case in that file where this bug came up, so I'm going to close this issue since I can't see any other immediate cases where this would cause a problem. It might also be a good idea to add some tests verifying that enum.Enum multiple inheritance works as expected for cases like this to Lib/test/test_enum.py.

---

For anyone that needs a quick fix for python < 3.9, the following based on the change should work (I sure hope this doesn't break something):

from typing import Generic

if hasattr(Generic, "__new__"):
    del Generic.__new__

Alternatively, the following works more generally if the mixin __new__ is completely unnecessary:

class Bar(Foo[int], Enum):
    __new__ = object.__new__

[ethanfurman]

@SimpleArt : Your last solution is the better one.

To detail why the error happens:

• EnumMeta looks for the __new__ to use to create the members, and finds Generic.__new__
• EnumMeta calls Generic.__new__
• Generic.__new__ then calls super().__new__
• which calls Enum.__new__
• which should not be called during member creation

So, a composite Enum needs to make sure that the __new__ used to create the members is appropriate, and if it isn't then it must provide one. In the case of Generic (in 3.8), simply using object.__new__ instead is sufficient. In other cases you may need to write a __new__ that calls any parents necessary.