Skip to content

Commit a7d4c67

Browse files
hauntsaninjaIvan Levkivskyi
authored and
Ivan Levkivskyi
committed
Document PEP 585, 563, 604 and more (#9763)
Fixes #8629, fixes #8523. This creates a new page to document issues arising from discrepancies between the runtime and annotations. I felt this was better, rather than force-fitting things into existing pages and "common issues", for instance, it prevents us from having to explain PEP 563 in several different places. I do still list the runtime errors you'd get in the "common issues" page to preserve SEO :-) "String literal types", "Class name forward references", and "Import cycles" are basically the same as where they were copied over from. This also factors out the documentation of PEP 604 that I promised when merging that PR (it seemed pretty verbose, particularly for the "kinds of types" page). It's also a good place to document PEP 613, when we get around to supporting that. Resolves #9856. Co-authored-by: hauntsaninja <>
1 parent 28f92ac commit a7d4c67

File tree

5 files changed

+361
-259
lines changed

5 files changed

+361
-259
lines changed

docs/source/builtin_types.rst

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,3 +38,8 @@ though they are similar to abstract base classes defined in
3838
:py:mod:`collections.abc` (formerly ``collections``), they are not identical. In
3939
particular, prior to Python 3.9, the built-in collection type objects do not
4040
support indexing.
41+
42+
In Python 3.9 and later, built-in collection type objects support indexing. This
43+
means that you can use built-in classes or those from :py:mod:`collections.abc`
44+
instead of importing from :py:mod:`typing`. See :ref:`generic-builtins` for more
45+
details.

docs/source/common_issues.rst

Lines changed: 15 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -210,6 +210,21 @@ checking would require a large number of ``assert foo is not None``
210210
checks to be inserted, and you want to minimize the number
211211
of code changes required to get a clean mypy run.
212212

213+
Issues with code at runtime
214+
---------------------------
215+
216+
Idiomatic use of type annotations can sometimes run up against what a given
217+
version of Python considers legal code. These can result in some of the
218+
following errors when trying to run your code:
219+
220+
* ``ImportError`` from circular imports
221+
* ``NameError: name 'X' is not defined`` from forward references
222+
* ``TypeError: 'type' object is not subscriptable`` from types that are not generic at runtime
223+
* ``ImportError`` or ``ModuleNotFoundError`` from use of stub definitions not available at runtime
224+
* ``TypeError: unsupported operand type(s) for |: 'type' and 'type'`` from use of new syntax
225+
226+
For dealing with these, see :ref:`runtime_troubles`.
227+
213228
Mypy runs are slow
214229
------------------
215230

@@ -499,112 +514,6 @@ to see the types of all local variables at once. Example:
499514
run your code. Both are always available and you don't need to import
500515
them.
501516

502-
503-
.. _import-cycles:
504-
505-
Import cycles
506-
-------------
507-
508-
An import cycle occurs where module A imports module B and module B
509-
imports module A (perhaps indirectly, e.g. ``A -> B -> C -> A``).
510-
Sometimes in order to add type annotations you have to add extra
511-
imports to a module and those imports cause cycles that didn't exist
512-
before. If those cycles become a problem when running your program,
513-
there's a trick: if the import is only needed for type annotations in
514-
forward references (string literals) or comments, you can write the
515-
imports inside ``if TYPE_CHECKING:`` so that they are not executed at runtime.
516-
Example:
517-
518-
File ``foo.py``:
519-
520-
.. code-block:: python
521-
522-
from typing import List, TYPE_CHECKING
523-
524-
if TYPE_CHECKING:
525-
import bar
526-
527-
def listify(arg: 'bar.BarClass') -> 'List[bar.BarClass]':
528-
return [arg]
529-
530-
File ``bar.py``:
531-
532-
.. code-block:: python
533-
534-
from typing import List
535-
from foo import listify
536-
537-
class BarClass:
538-
def listifyme(self) -> 'List[BarClass]':
539-
return listify(self)
540-
541-
.. note::
542-
543-
The :py:data:`~typing.TYPE_CHECKING` constant defined by the :py:mod:`typing` module
544-
is ``False`` at runtime but ``True`` while type checking.
545-
546-
Python 3.5.1 doesn't have :py:data:`~typing.TYPE_CHECKING`. An alternative is
547-
to define a constant named ``MYPY`` that has the value ``False``
548-
at runtime. Mypy considers it to be ``True`` when type checking.
549-
Here's the above example modified to use ``MYPY``:
550-
551-
.. code-block:: python
552-
553-
from typing import List
554-
555-
MYPY = False
556-
if MYPY:
557-
import bar
558-
559-
def listify(arg: 'bar.BarClass') -> 'List[bar.BarClass]':
560-
return [arg]
561-
562-
.. _not-generic-runtime:
563-
564-
Using classes that are generic in stubs but not at runtime
565-
----------------------------------------------------------
566-
567-
Some classes are declared as generic in stubs, but not at runtime. Examples
568-
in the standard library include :py:class:`os.PathLike` and :py:class:`queue.Queue`.
569-
Subscripting such a class will result in a runtime error:
570-
571-
.. code-block:: python
572-
573-
from queue import Queue
574-
575-
class Tasks(Queue[str]): # TypeError: 'type' object is not subscriptable
576-
...
577-
578-
results: Queue[int] = Queue() # TypeError: 'type' object is not subscriptable
579-
580-
To avoid these errors while still having precise types you can either use
581-
string literal types or :py:data:`~typing.TYPE_CHECKING`:
582-
583-
.. code-block:: python
584-
585-
from queue import Queue
586-
from typing import TYPE_CHECKING
587-
588-
if TYPE_CHECKING:
589-
BaseQueue = Queue[str] # this is only processed by mypy
590-
else:
591-
BaseQueue = Queue # this is not seen by mypy but will be executed at runtime.
592-
593-
class Tasks(BaseQueue): # OK
594-
...
595-
596-
results: 'Queue[int]' = Queue() # OK
597-
598-
If you are running Python 3.7+ you can use ``from __future__ import annotations``
599-
as a (nicer) alternative to string quotes, read more in :pep:`563`. For example:
600-
601-
.. code-block:: python
602-
603-
from __future__ import annotations
604-
from queue import Queue
605-
606-
results: Queue[int] = Queue() # This works at runtime
607-
608517
.. _silencing-linters:
609518

610519
Silencing linters

docs/source/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ Mypy is a static type checker for Python 3 and Python 2.7.
3535
type_inference_and_annotations
3636
kinds_of_types
3737
class_basics
38+
runtime_troubles
3839
protocols
3940
python2
4041
dynamic_typing

docs/source/kinds_of_types.rst

Lines changed: 8 additions & 153 deletions
Original file line numberDiff line numberDiff line change
@@ -243,93 +243,24 @@ more specific type:
243243

244244
.. _alternative_union_syntax:
245245

246-
Alternative union syntax
247-
------------------------
246+
X | Y syntax for Unions
247+
-----------------------
248248

249-
`PEP 604 <https://www.python.org/dev/peps/pep-0604/>`_ introduced an alternative way
250-
for writing union types. Starting with **Python 3.10** it is possible to write
251-
``Union[int, str]`` as ``int | str``. Any of the following options is possible
249+
:pep:`604` introduced an alternative way for spelling union types. In Python
250+
3.10 and later, you can write ``Union[int, str]`` as ``int | str``. It is
251+
possible to use this syntax in versions of Python where it isn't supported by
252+
the runtime with some limitations, see :ref:`runtime_troubles`.
252253

253254
.. code-block:: python
254255
255256
from typing import List
256257
257-
# Use as Union
258258
t1: int | str # equivalent to Union[int, str]
259259
260-
# Use as Optional
261260
t2: int | None # equivalent to Optional[int]
262261
263-
# Use in generics
264-
t3: List[int | str] # equivalent to List[Union[int, str]]
265-
266-
# Use in type aliases
267-
T4 = int | None
268-
x: T4
269-
270-
# Quoted variable annotations
271-
t5: "int | str"
272-
273-
# Quoted function annotations
274-
def f(t6: "int | str") -> None: ...
275-
276-
# Type comments
277-
t6 = 42 # type: int | str
278-
279-
It is possible to use most of these even for earlier versions. However there are some
280-
limitations to be aware of.
281-
282-
.. _alternative_union_syntax_stub_files:
283-
284-
Stub files
285-
""""""""""
286-
287-
All options are supported, regardless of the Python version the project uses.
288-
289-
.. _alternative_union_syntax_37:
290-
291-
Python 3.7 - 3.9
292-
""""""""""""""""
293-
294-
It is necessary to add ``from __future__ import annotations`` to delay the evaluation
295-
of type annotations. Not using it would result in a ``TypeError``.
296-
This does not apply for **type comments**, **quoted function** and **quoted variable** annotations,
297-
as those also work for earlier versions, see :ref:`below <alternative_union_syntax_older_version>`.
298-
299-
.. warning::
300-
301-
Type aliases are **NOT** supported! Those result in a ``TypeError`` regardless
302-
if the evaluation of type annotations is delayed.
303-
304-
Dynamic evaluation of annotations is **NOT** possible (e.g. ``typing.get_type_hints`` and ``eval``).
305-
See `note PEP 604 <https://www.python.org/dev/peps/pep-0604/#change-only-pep-484-type-hints-to-accept-the-syntax-type1-type2>`_.
306-
Use ``typing.Union`` or **Python 3.10** instead if you need those!
307-
308-
.. code-block:: python
309-
310-
from __future__ import annotations
311-
312-
t1: int | None
313-
314-
# Type aliases
315-
T2 = int | None # TypeError!
316-
317-
.. _alternative_union_syntax_older_version:
318-
319-
Older versions
320-
""""""""""""""
321-
322-
+------------------------------------------+-----------+-----------+-----------+
323-
| Python Version | 3.6 | 3.0 - 3.5 | 2.7 |
324-
+==========================================+===========+===========+===========+
325-
| Type comments | yes | yes | yes |
326-
+------------------------------------------+-----------+-----------+-----------+
327-
| Quoted function annotations | yes | yes | |
328-
+------------------------------------------+-----------+-----------+-----------+
329-
| Quoted variable annotations | yes | | |
330-
+------------------------------------------+-----------+-----------+-----------+
331-
| Everything else | | | |
332-
+------------------------------------------+-----------+-----------+-----------+
262+
# Usable in type comments
263+
t3 = 42 # type: int | str
333264
334265
.. _strict_optional:
335266

@@ -565,82 +496,6 @@ valid for any type, but it's much more
565496
useful for a programmer who is reading the code. This also makes
566497
it easier to migrate to strict ``None`` checking in the future.
567498

568-
Class name forward references
569-
*****************************
570-
571-
Python does not allow references to a class object before the class is
572-
defined. Thus this code does not work as expected:
573-
574-
.. code-block:: python
575-
576-
def f(x: A) -> None: # Error: Name A not defined
577-
...
578-
579-
class A:
580-
...
581-
582-
In cases like these you can enter the type as a string literal — this
583-
is a *forward reference*:
584-
585-
.. code-block:: python
586-
587-
def f(x: 'A') -> None: # OK
588-
...
589-
590-
class A:
591-
...
592-
593-
Starting from Python 3.7 (:pep:`563`), you can add the special import ``from __future__ import annotations``,
594-
which makes the use of string literals in annotations unnecessary:
595-
596-
.. code-block:: python
597-
598-
from __future__ import annotations
599-
600-
def f(x: A) -> None: # OK
601-
...
602-
603-
class A:
604-
...
605-
606-
.. note::
607-
608-
Even with the ``__future__`` import, there are some scenarios that could still
609-
require string literals, typically involving use of forward references or generics in:
610-
611-
* :ref:`type aliases <type-aliases>`;
612-
* :ref:`casts <casts>`;
613-
* type definitions (see :py:class:`~typing.TypeVar`, :py:func:`~typing.NewType`, :py:class:`~typing.NamedTuple`);
614-
* base classes.
615-
616-
.. code-block:: python
617-
618-
# base class example
619-
class A(Tuple['B', 'C']): ... # OK
620-
class B: ...
621-
class C: ...
622-
623-
Of course, instead of using a string literal type or special import, you could move the
624-
function definition after the class definition. This is not always
625-
desirable or even possible, though.
626-
627-
Any type can be entered as a string literal, and you can combine
628-
string-literal types with non-string-literal types freely:
629-
630-
.. code-block:: python
631-
632-
def f(a: List['A']) -> None: ... # OK
633-
def g(n: 'int') -> None: ... # OK, though not useful
634-
635-
class A: pass
636-
637-
String literal types are never needed in ``# type:`` comments and :ref:`stub files <stub-files>`.
638-
639-
String literal types must be defined (or imported) later *in the same
640-
module*. They cannot be used to leave cross-module references
641-
unresolved. (For dealing with import cycles, see
642-
:ref:`import-cycles`.)
643-
644499
.. _type-aliases:
645500

646501
Type aliases

0 commit comments

Comments
 (0)