diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 1269bad5d70..0a9a0c69441 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -582,6 +582,7 @@ pep-0698.rst @jellezijlstra pep-0699.rst @Fidget-Spinner pep-0700.rst @pfmoore pep-0701.rst @pablogsal @isidentical @lysnikolaou +pep-0702.rst @jellezijlstra # ... # pep-0754.txt # ... diff --git a/pep-0702.rst b/pep-0702.rst new file mode 100644 index 00000000000..9c8089d8bb7 --- /dev/null +++ b/pep-0702.rst @@ -0,0 +1,308 @@ +PEP: 702 +Title: Marking deprecations using the type system +Author: Jelle Zijlstra +Status: Draft +Type: Standards Track +Topic: Typing +Content-Type: text/x-rst +Created: 30-Dec-2022 +Python-Version: 3.12 + + +Abstract +======== + +This PEP adds an ``@typing.deprecated()`` decorator that marks a class or function +as deprecated, enabling static checkers to warn when it is used. + +Motivation +========== + +As software evolves, new functionality is added and old functionality becomes +obsolete. Library developers want to work towards removing obsolete code while +giving their users time to migrate to new APIs. Python provides a mechanism for +achieving these goals: the :exc:`DeprecationWarning` warning class, which is +used to show warnings when deprecated functionality is used. This mechanism is +widely used: as of the writing of this PEP, the CPython main branch contains +about 150 distinct code paths that raise :exc:`!DeprecationWarning`. Many +third-party libraries also use :exc:`!DeprecationWarning` to mark deprecations. +In the `top 5000 PyPI packages `__, +there are: + +- 1911 matches for the regex ``warnings\.warn.*\bDeprecationWarning\b'``, + indicating use of :exc:`!DeprecationWarning` (not including cases where the + warning is split over multiple lines); +- 1661 matches for the regex ``^\s*@deprecated``, indicating use of some sort of + deprecation decorator. + +However, the current mechanism is often insufficient to ensure that users +of deprecated functionality update their code in time. For example, the +removal of various long-deprecated :mod:`unittest` features had to be +`reverted `__ +from Python 3.11 to give users more time to update their code. +Users may run their test suite with warnings disabled for practical reasons, +or deprecations may be triggered in code paths that are not covered by tests. + +Providing more ways for users to find out about deprecated functionality +can speed up the migration process. This PEP proposes to leverage static type +checkers to communicate deprecations to users. Such checkers have a thorough +semantic understanding of user code, enabling them to detect and report +deprecations that a single ``grep`` invocation could not find. In addition, many type +checkers integrate with IDEs, enabling users to see deprecation warnings +right in their editors. + +Rationale +========= + +At first glance, deprecations may not seem like a topic that type checkers should +touch. After all, type checkers are concerned with checking whether code will +work as is, not with potential future changes. However, the analysis that type +checkers perform on code to find type errors is very similar to the analysis +that would be needed to detect usage of many deprecations. Therefore, type +checkers are well placed to find and report deprecations. + +Other languages already have similar functionality: + +* GCC supports a ``deprecated`` `attribute `__ + on function declarations. This powers CPython's ``Py_DEPRECATED`` macro. +* GraphQL `supports `__ + marking fields as ``@deprecated``. +* Kotlin `supports `__ + a ``Deprecated`` annotation. +* Scala `supports `__ + an ``@deprecated`` annotation. +* Swift `supports `__ + using the ``@available`` attribute to mark APIs as deprecated. +* TypeScript `uses `__ + the ``@deprecated`` JSDoc tag to issue a hint marking use of + deprecated functionality. + +Several users have requested support for such a feature: + +* `typing-sig thread `__ +* `Pyright feature request `__ +* `mypy feature request `__ + + +Specification +============= + +A new decorator ``@deprecated()`` is added to the :mod:`typing` module. This +decorator can be used on a class, function or method to mark it as deprecated. +This includes :class:`typing.TypedDict` and :class:`typing.NamedTuple` definitions. +With overloaded functions, the decorator may be applied to individual overloads, +indicating that the particular overload is deprecated. The decorator may also be +applied to the overload implementation function, indicating that the entire function +is deprecated. + +The decorator takes a single argument of type ``str``, which is a message that should +be shown by the type checker when it encounters a usage of the decorated object. +The message must be a string literal. +The content of deprecation messages is up to the user, but it may include the version +in which the deprecated object is to be removed, and information about suggested +replacement APIs. + +Type checkers should produce a diagnostic whenever they encounter a usage of an +object marked as deprecated. For deprecated overloads, this includes all calls +that resolve to the deprecated overload. +For deprecated classes and functions, this includes: + +* References through module, class, or instance attributes (``module.deprecated_object``, + ``module.SomeClass.deprecated_method``, ``module.SomeClass().deprecated_method``) +* Any usage of deprecated objects in their defining module + (``x = deprecated_object()`` in ``module.py``) +* If ``import *`` is used, usage of deprecated objects from the + module (``from module import *; x = deprecated_object()``) +* ``from`` imports (``from module import deprecated_object``) + +There are some additional scenarios where deprecations could come into play: + +* An object implements a :class:`typing.Protocol`, but one of the methods + required for protocol compliance is deprecated. +* A class uses the ``@override`` decorator from :pep:`698` to assert that + its method overrides a base class method, but the base class method is + deprecated. + +As these scenarios appear complex and relatively unlikely to come up in practice, +this PEP does not mandate that type checkers detect them. + +Example +------- + +As an example, consider this library stub named ``library.pyi``: + +.. code-block:: python + + from typing import deprecated + + @deprecated("Use Spam instead") + class Ham: ... + + @deprecated("It is pining for the fiords") + def norwegian_gray(x: int) -> int: ... + + @overload + @deprecated("Only str will be allowed") + def foo(x: int) -> str: ... + @overload + def foo(x: str) -> str: ... + +Here is how type checkers should handle usage of this library: + +.. code-block:: python + + from library import Ham # error: Use of deprecated class Ham. Use Spam instead. + + import library + + library.norwegian_gray(1) # error: Use of deprecated function norwegian_gray. It is pining for the fiords. + map(library.norwegian_gray, [1, 2, 3]) # error: Use of deprecated function norwegian_gray. It is pining for the fiords. + + library.foo(1) # error: Use of deprecated overload for foo. Only str will be allowed. + library.foo("x") # no error + +Runtime behavior +---------------- + +At runtime, the decorator sets an attribute ``__deprecated__`` on the decorated +object. The value of the attribute is the message passed to the decorator. +The decorator returns the original object. Notably, it does not issue a runtime +:exc:`DeprecationWarning`. + +For compatibility with :func:`typing.get_overloads`, the ``@deprecated`` +decorator should be placed after the ``@overload`` decorator. + +Type checker behavior +--------------------- + +This PEP does not specify exactly how type checkers should present deprecation +diagnostics to their users. However, some users (e.g., application developers +targeting only a specific version of Python) may not care about deprecations, +while others (e.g., library developers who want their library to remain +compatible with future versions of Python) would want to catch any use of +deprecated functionality in their CI pipeline. Therefore, it is recommended +that type checkers provide configuration options that cover both use cases. +As with any other type checker error, it is also possible to ignore deprecations +using ``# type: ignore`` comments. + +Deprecation policy +------------------ + +We propose that CPython's deprecation policy (:pep:`387`) is updated to require that new deprecations +use the functionality in this PEP to alert users +about the deprecation, if possible. Concretely, this means that new +deprecations should be accompanied by a change to the ``typeshed`` repo to +add the ``@deprecated`` decorator in the appropriate place. +This requirement does not apply to deprecations that cannot be expressed +using this PEP's functionality. + +Backwards compatibility +======================= + +Creating a new decorator poses no backwards compatibility concerns. +As with all new typing functionality, the ``@deprecated`` decorator +will be added to the ``typing_extensions`` module, enabling its use +in older versions of Python. + +How to teach this +================= + +For users who encounter deprecation warnings in their IDE or type +checker output, the messages they receive should be clear and self-explanatory. +Usage of the ``@deprecated`` decorator will be an advanced feature +mostly relevant to library authors. The decorator should be mentioned +in relevant documentation (e.g., :pep:`387` and the :exc:`DeprecationWarning` +documentation) as an additional way to mark deprecated functionality. + +Reference implementation +======================== + +A runtime implementation of the ``@deprecated`` decorator is +`available `__. +The ``pyanalyze`` type checker has +`prototype support `__ +for emitting deprecation errors. + +Rejected ideas +============== + +Runtime warnings +---------------- + +Users might expect usage of the ``@deprecated`` decorator to issue a +:exc:`DeprecationWarning` at runtime. However, this would raise a number of +thorny issues: + +* When the decorator is applied to a class or an overload, the warning + would not be raised as expected. +* Users may want to control the ``warn`` call in more detail (e.g., + changing the warning class). +* ``typing.py`` generally aims to avoid affecting runtime behavior. + +Users who want to use ``@deprecated`` while also issuing a runtime warning +can use the ``if TYPE_CHECKING:`` idiom, for example: + +.. code-block:: python + + from typing import TYPE_CHECKING + import functools + import warnings + + if TYPE_CHECKING: + from typing import deprecated + else: + def deprecated(msg): + def decorator(func): + @functools.wraps(func) + def wrapper(*args, **kwargs): + warnings.warn(msg, DeprecationWarning, stacklevel=2) + return func(*args, **kwargs) + wrapper.__deprecated__ = msg + return wrapper + return decorator + +Deprecation of modules and attributes +------------------------------------- + +This PEP covers deprecations of classes, functions and overloads. This +allows type checkers to detect many but not all possible deprecations. +To evaluate whether additional functionality would be worthwhile, I +`examined `__ +all current deprecations in the CPython standard library. + +I found: + +* 74 deprecations of functions, methods and classes (supported by this PEP) +* 28 deprecations of whole modules (largely due to :pep:`594`) +* 9 deprecations of function parameters (supported by this PEP through + decorating overloads) +* 1 deprecation of a constant +* 38 deprecations that are not easily detectable in the type system (for + example, for calling :func:`asyncio.get_event_loop` without an active + event loop) + +Modules could be marked as deprecated by adding a ``__deprecated__`` +module-level constant. However, the need for this is limited, and it +is relatively easy to detect usage of deprecated modules simply by +grepping. Therefore, this PEP omits support for whole-module deprecations. +As a workaround, users could mark all module-level classes and functions +with ``@deprecated``. + +For deprecating module-level constants, object attributes, and function +parameters, a ``Deprecated[type, message]`` type modifier, similar to +``Annotated`` could be added. However, this would create a new place +in the type system where strings are just strings, not forward references, +complicating the implementation of type checkers. In addition, my data +show that this feature is not commonly needed. + +Acknowledgments +=============== + +A call with the typing-sig meetup group led to useful feedback on this +proposal. + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive.