PEP 604 Union syntax does not support forward references

[TNThung]

BPO	45857
Nosy	@gvanrossum, @ericvsmith, @Fidget-Spinner, @AlexWaygood

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-11-21.14:48:31.479>
labels = ['type-bug', '3.10', '3.11']
title = 'PEP 604 Union syntax does not support forward references'
updated_at = <Date 2021-11-22.16:46:08.109>
user = 'https://bugs.python.org/TNThung'

bugs.python.org fields:

activity = <Date 2021-11-22.16:46:08.109>
actor = 'AlexWaygood'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2021-11-21.14:48:31.479>
creator = 'TNThung'
dependencies = []
files = []
hgrepos = []
issue_num = 45857
keywords = []
message_count = 6.0
messages = ['406720', '406724', '406726', '406731', '406775', '406792']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'eric.smith', 'kj', 'AlexWaygood', 'TNThung']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45857'
versions = ['Python 3.10', 'Python 3.11']

Linked PRs

• gh-90015: Document that PEP-604 unions do not support forward references #105366
• [3.12] gh-90015: Document that PEP-604 unions do not support forward references (GH-105366) #105460
• [3.11] gh-90015: Document that PEP-604 unions do not support forward references (GH-105366) #105461

[TNThung]

The class methods have a problem compiling when the type refers to the union of itself and others.

# tmp.py
class Foo:
    def __init__(self, tmp: "Foo"|int):
        pass

# Error
Traceback (most recent call last):
  File "/Project/Mslc/Grammar/tmp.py", line 1, in <module>
    class Foo:
  File "/Project/Mslc/Grammar/tmp.py", line 2, in Foo
    def __init__(self, tmp: "Foo"|int):
TypeError: unsupported operand type(s) for |: 'str' and 'type'

[AlexWaygood]

Reproduced on 3.11. The error occurs if a type is on the left-hand-side of the operand, as well as if a type is on the right-hand-side:

>>> int | "str"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for |: 'type' and 'str'
>>> "str" | int
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for |: 'str' and 'type'

[ericvsmith]

Presumably the correct way to do this is:

def __init__(self, tmp: "Foo|int"):

That is, the entire type hint is a string.

[AlexWaygood]

Arguably, either the implementation should be altered to support forward references, or the documentation at https://docs.python.org/3/library/stdtypes.html#union-type should be altered to make clear that, when type-hinting a union that includes a forward reference, the entire expression should be given as a string, as Eric suggests.

[Fidget-Spinner]

I think I saw a similar bug report elsewhere (or maybe I'm misremembering). Anyways, Eric is right, the correct way is to wrap the entire thing, so "Foo|int" instead of "Foo"|int.

@alex you brought up some good suggestions, I'll try to address them:

Arguably, either the implementation should be altered to support forward references

Unfortunately that's more complex than it seems. The original draft PEP-604 proposed implementation actually imported Union from typing.py, and I recall Guido disliking the idea that a builtin type should depend on typing.py. I have to agree with that philosophy here. I also don't think the alternative -- implementing a builtin ForwardRef type isn't worth the complexity unless our situation changes.

the documentation at https://docs.python.org/3/library/stdtypes.html#union-type should be altered to make clear that ...

The first line says: "A union object holds the value of the | (bitwise or) operation on multiple type objects." It says *type objects*, which strings don't belong to.

@tnthung does Eric's suggestion work for you? Or do you need something else?

[AlexWaygood]

Thanks, Ken! To clarify: I agree that changing the implementation here would probably be a bad way to go: it would be foolish to try to replicate all the functionality of the typing module as builtins. I also think the existing documentation at https://docs.python.org/3/library/stdtypes.html#union-type is actually very good, so I don't think it needs a fundamental rewrite by any means.

I do still think a sentence or two could be added to the documentation, however, clarifying how to deal with forward references, since the behaviour here is different to the older, more established syntax using typing.Union. Something like this?

"""
Note: The object on both sides of the | operand must be an object that defines the or special method. As the str type does not support or, the expression int | "Foo", where "Foo" is a reference to a class not yet defined, will fail at runtime. To annotate forward references using union-type expressions, present the whole expression as a string, e.g. "int | Foo".
"""

[pbryan]

I have a related question about how type aliases should work.

Starting with Union with a forward reference as a type alias:

>>> from typing import Union
>>> Alias = Union[str, "Foo"]
>>> Alias
typing.Union[str, ForwardRef('Foo')]

With the new UnionType, the following is not valid:

>>> Alias = str | "Foo"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for |: 'type' and 'str'

The following is also invalid:

>>> from typing import ForwardRef
>>> Alias = str | ForwardRef("Foo")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for |: 'type' and 'ForwardRef' 

Putting everything in a string seems wrong to me, because it's not evaluated as a type:

>>> Alias = "str | Foo"
>>> Alias
'str | Foo'

I would suggest enhancing ForwardRef.__or__ to be compatible with UnionType, so that the following would be valid:

>>> from typing import ForwardRef
>>> Alias = str | ForwardRef("Foo")
>>> Alias
str | ForwardRef('Foo')

Thoughts?

[gvanrossum]

You should never use ForwardRef -- it was intended for internal use only. The docs say "This class should not be instantiated by a user, but may be used by introspection tools." (By the latter use, they mean things like isinstance(x, typing.ForwardRef).)

Instead, use TypeAlias to explicitly declare Alias to be a type alias, like this:

from typing import TypeAlias
Alias: TypeAlias = 'str | Foo'

[pbryan]

Instead, use TypeAlias to explicitly declare Alias to be a type alias, like this:

Ah, thanks.

[pbryan]

Is there some way to evaluate TypeAlias as a type alias at runtime?

[gvanrossum]

Is there some way to evaluate TypeAlias as a type alias at runtime?

Alas not. I think earlier in the thread it was suggested that it would be a bad idea to change the runtime implementation, so we're probably stuck with this, unless you have a compelling (i.e., real-world) use case that needs to handle this.

[pbryan]

The easy workaround is to continue to use Union for now. I'll hold off on making a case to see what changes to string annotations occur (e.g. PEP 649).

[AlexWaygood]

We've now documented that PEP-604 aliases do not support forward references at runtime.

@pbryan, you may be interested in the new typing.TypeAliasType, part of PEP-695. It natively supports forward references, which are evaluated only if you access the __value__ property.

https://docs.python.org/3.13/library/typing.html#type-aliases