-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
PEP 702: Type system support for deprecations #2942
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
ad12b96
PEP 702: Type system support for deprecations
JelleZijlstra 423bb10
fix headers
JelleZijlstra ea6c13d
fix better
JelleZijlstra e6a3809
it's 702
JelleZijlstra 70ebe5e
maybe this
JelleZijlstra 6dab124
fix more lists
JelleZijlstra c1ad77d
message must be a literal
JelleZijlstra 1889981
Update pep-0702.rst
JelleZijlstra 7706fcf
Update pep-0702.rst
JelleZijlstra 9f69f85
Update pep-0702.rst
JelleZijlstra 98aa4fc
Apply suggestions from code review
JelleZijlstra 60f7d06
round of changes
JelleZijlstra f78cbde
it was a call
JelleZijlstra 5db4a0b
add a line about type ignore
JelleZijlstra d5e8168
some additions
JelleZijlstra 0732966
thanks Hugo
JelleZijlstra 1f706a0
fix list
JelleZijlstra 23ab995
Update pep-0702.rst
JelleZijlstra c4316c6
a few more changes
JelleZijlstra File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,308 @@ | ||
PEP: 702 | ||
Title: Marking deprecations using the type system | ||
Author: Jelle Zijlstra <[email protected]> | ||
Status: Draft | ||
Type: Standards Track | ||
Topic: Typing | ||
Content-Type: text/x-rst | ||
Created: 30-Dec-2022 | ||
Python-Version: 3.12 | ||
|
||
|
||
Abstract | ||
======== | ||
|
||
This PEP adds an ``@typing.deprecated()`` decorator that marks a class or function | ||
as deprecated, enabling static checkers to warn when it is used. | ||
|
||
Motivation | ||
========== | ||
|
||
As software evolves, new functionality is added and old functionality becomes | ||
obsolete. Library developers want to work towards removing obsolete code while | ||
giving their users time to migrate to new APIs. Python provides a mechanism for | ||
achieving these goals: the :exc:`DeprecationWarning` warning class, which is | ||
used to show warnings when deprecated functionality is used. This mechanism is | ||
widely used: as of the writing of this PEP, the CPython main branch contains | ||
about 150 distinct code paths that raise :exc:`!DeprecationWarning`. Many | ||
third-party libraries also use :exc:`!DeprecationWarning` to mark deprecations. | ||
In the `top 5000 PyPI packages <https://dev.to/hugovk/how-to-search-5000-python-projects-31gk>`__, | ||
there are: | ||
|
||
- 1911 matches for the regex ``warnings\.warn.*\bDeprecationWarning\b'``, | ||
indicating use of :exc:`!DeprecationWarning` (not including cases where the | ||
warning is split over multiple lines); | ||
- 1661 matches for the regex ``^\s*@deprecated``, indicating use of some sort of | ||
deprecation decorator. | ||
|
||
However, the current mechanism is often insufficient to ensure that users | ||
of deprecated functionality update their code in time. For example, the | ||
removal of various long-deprecated :mod:`unittest` features had to be | ||
`reverted <https://github.com/python/cpython/commit/b50322d20337ca468f2070eedb051a16ee1eba94>`__ | ||
from Python 3.11 to give users more time to update their code. | ||
Users may run their test suite with warnings disabled for practical reasons, | ||
or deprecations may be triggered in code paths that are not covered by tests. | ||
|
||
Providing more ways for users to find out about deprecated functionality | ||
can speed up the migration process. This PEP proposes to leverage static type | ||
checkers to communicate deprecations to users. Such checkers have a thorough | ||
semantic understanding of user code, enabling them to detect and report | ||
deprecations that a single ``grep`` invocation could not find. In addition, many type | ||
checkers integrate with IDEs, enabling users to see deprecation warnings | ||
right in their editors. | ||
|
||
Rationale | ||
========= | ||
|
||
At first glance, deprecations may not seem like a topic that type checkers should | ||
touch. After all, type checkers are concerned with checking whether code will | ||
work as is, not with potential future changes. However, the analysis that type | ||
checkers perform on code to find type errors is very similar to the analysis | ||
that would be needed to detect usage of many deprecations. Therefore, type | ||
checkers are well placed to find and report deprecations. | ||
|
||
Other languages already have similar functionality: | ||
|
||
* GCC supports a ``deprecated`` `attribute <https://gcc.gnu.org/onlinedocs/gcc-3.1.1/gcc/Type-Attributes.html>`__ | ||
on function declarations. This powers CPython's ``Py_DEPRECATED`` macro. | ||
* GraphQL `supports <https://spec.graphql.org/June2018/#sec-Field-Deprecation>`__ | ||
marking fields as ``@deprecated``. | ||
* Kotlin `supports <https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-deprecated/>`__ | ||
a ``Deprecated`` annotation. | ||
* Scala `supports <https://www.scala-lang.org/api/2.12.5/scala/deprecated.html>`__ | ||
an ``@deprecated`` annotation. | ||
* Swift `supports <https://docs.swift.org/swift-book/ReferenceManual/Attributes.html>`__ | ||
using the ``@available`` attribute to mark APIs as deprecated. | ||
* TypeScript `uses <https://www.typescriptlang.org/docs/handbook/jsdoc-supported-types.html#deprecated>`__ | ||
the ``@deprecated`` JSDoc tag to issue a hint marking use of | ||
deprecated functionality. | ||
|
||
Several users have requested support for such a feature: | ||
|
||
* `typing-sig thread <https://mail.python.org/archives/list/[email protected]/thread/E24WTMQUTGKPFKEXVCGGEFFMG7LDF3WT/>`__ | ||
* `Pyright feature request <https://github.com/microsoft/pyright/discussions/2300>`__ | ||
* `mypy feature request <https://github.com/python/mypy/issues/11439>`__ | ||
|
||
|
||
Specification | ||
============= | ||
|
||
A new decorator ``@deprecated()`` is added to the :mod:`typing` module. This | ||
decorator can be used on a class, function or method to mark it as deprecated. | ||
This includes :class:`typing.TypedDict` and :class:`typing.NamedTuple` definitions. | ||
With overloaded functions, the decorator may be applied to individual overloads, | ||
indicating that the particular overload is deprecated. The decorator may also be | ||
applied to the overload implementation function, indicating that the entire function | ||
is deprecated. | ||
|
||
The decorator takes a single argument of type ``str``, which is a message that should | ||
JelleZijlstra marked this conversation as resolved.
Show resolved
Hide resolved
|
||
be shown by the type checker when it encounters a usage of the decorated object. | ||
The message must be a string literal. | ||
The content of deprecation messages is up to the user, but it may include the version | ||
in which the deprecated object is to be removed, and information about suggested | ||
replacement APIs. | ||
|
||
Type checkers should produce a diagnostic whenever they encounter a usage of an | ||
object marked as deprecated. For deprecated overloads, this includes all calls | ||
that resolve to the deprecated overload. | ||
For deprecated classes and functions, this includes: | ||
|
||
* References through module, class, or instance attributes (``module.deprecated_object``, | ||
``module.SomeClass.deprecated_method``, ``module.SomeClass().deprecated_method``) | ||
* Any usage of deprecated objects in their defining module | ||
(``x = deprecated_object()`` in ``module.py``) | ||
JelleZijlstra marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* If ``import *`` is used, usage of deprecated objects from the | ||
module (``from module import *; x = deprecated_object()``) | ||
* ``from`` imports (``from module import deprecated_object``) | ||
|
||
There are some additional scenarios where deprecations could come into play: | ||
|
||
* An object implements a :class:`typing.Protocol`, but one of the methods | ||
required for protocol compliance is deprecated. | ||
* A class uses the ``@override`` decorator from :pep:`698` to assert that | ||
its method overrides a base class method, but the base class method is | ||
deprecated. | ||
|
||
As these scenarios appear complex and relatively unlikely to come up in practice, | ||
this PEP does not mandate that type checkers detect them. | ||
|
||
Example | ||
------- | ||
|
||
As an example, consider this library stub named ``library.pyi``: | ||
|
||
.. code-block:: python | ||
|
||
from typing import deprecated | ||
|
||
@deprecated("Use Spam instead") | ||
class Ham: ... | ||
|
||
@deprecated("It is pining for the fiords") | ||
def norwegian_gray(x: int) -> int: ... | ||
|
||
@overload | ||
@deprecated("Only str will be allowed") | ||
def foo(x: int) -> str: ... | ||
@overload | ||
CAM-Gerlach marked this conversation as resolved.
Show resolved
Hide resolved
|
||
def foo(x: str) -> str: ... | ||
|
||
Here is how type checkers should handle usage of this library: | ||
|
||
.. code-block:: python | ||
|
||
from library import Ham # error: Use of deprecated class Ham. Use Spam instead. | ||
|
||
import library | ||
|
||
library.norwegian_gray(1) # error: Use of deprecated function norwegian_gray. It is pining for the fiords. | ||
map(library.norwegian_gray, [1, 2, 3]) # error: Use of deprecated function norwegian_gray. It is pining for the fiords. | ||
|
||
library.foo(1) # error: Use of deprecated overload for foo. Only str will be allowed. | ||
library.foo("x") # no error | ||
|
||
Runtime behavior | ||
---------------- | ||
|
||
At runtime, the decorator sets an attribute ``__deprecated__`` on the decorated | ||
object. The value of the attribute is the message passed to the decorator. | ||
The decorator returns the original object. Notably, it does not issue a runtime | ||
JelleZijlstra marked this conversation as resolved.
Show resolved
Hide resolved
|
||
:exc:`DeprecationWarning`. | ||
|
||
For compatibility with :func:`typing.get_overloads`, the ``@deprecated`` | ||
decorator should be placed after the ``@overload`` decorator. | ||
|
||
Type checker behavior | ||
--------------------- | ||
|
||
This PEP does not specify exactly how type checkers should present deprecation | ||
diagnostics to their users. However, some users (e.g., application developers | ||
targeting only a specific version of Python) may not care about deprecations, | ||
while others (e.g., library developers who want their library to remain | ||
compatible with future versions of Python) would want to catch any use of | ||
deprecated functionality in their CI pipeline. Therefore, it is recommended | ||
that type checkers provide configuration options that cover both use cases. | ||
As with any other type checker error, it is also possible to ignore deprecations | ||
using ``# type: ignore`` comments. | ||
|
||
Deprecation policy | ||
------------------ | ||
|
||
We propose that CPython's deprecation policy (:pep:`387`) is updated to require that new deprecations | ||
use the functionality in this PEP to alert users | ||
about the deprecation, if possible. Concretely, this means that new | ||
deprecations should be accompanied by a change to the ``typeshed`` repo to | ||
add the ``@deprecated`` decorator in the appropriate place. | ||
This requirement does not apply to deprecations that cannot be expressed | ||
using this PEP's functionality. | ||
|
||
Backwards compatibility | ||
======================= | ||
|
||
Creating a new decorator poses no backwards compatibility concerns. | ||
As with all new typing functionality, the ``@deprecated`` decorator | ||
will be added to the ``typing_extensions`` module, enabling its use | ||
in older versions of Python. | ||
|
||
How to teach this | ||
================= | ||
|
||
For users who encounter deprecation warnings in their IDE or type | ||
checker output, the messages they receive should be clear and self-explanatory. | ||
Usage of the ``@deprecated`` decorator will be an advanced feature | ||
mostly relevant to library authors. The decorator should be mentioned | ||
in relevant documentation (e.g., :pep:`387` and the :exc:`DeprecationWarning` | ||
JelleZijlstra marked this conversation as resolved.
Show resolved
Hide resolved
|
||
documentation) as an additional way to mark deprecated functionality. | ||
|
||
Reference implementation | ||
======================== | ||
|
||
A runtime implementation of the ``@deprecated`` decorator is | ||
`available <https://github.com/python/typing_extensions/pull/105>`__. | ||
The ``pyanalyze`` type checker has | ||
`prototype support <https://github.com/quora/pyanalyze/pull/578>`__ | ||
for emitting deprecation errors. | ||
|
||
Rejected ideas | ||
============== | ||
|
||
Runtime warnings | ||
---------------- | ||
|
||
Users might expect usage of the ``@deprecated`` decorator to issue a | ||
:exc:`DeprecationWarning` at runtime. However, this would raise a number of | ||
thorny issues: | ||
|
||
* When the decorator is applied to a class or an overload, the warning | ||
would not be raised as expected. | ||
JelleZijlstra marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* Users may want to control the ``warn`` call in more detail (e.g., | ||
changing the warning class). | ||
JelleZijlstra marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* ``typing.py`` generally aims to avoid affecting runtime behavior. | ||
|
||
Users who want to use ``@deprecated`` while also issuing a runtime warning | ||
can use the ``if TYPE_CHECKING:`` idiom, for example: | ||
JelleZijlstra marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
.. code-block:: python | ||
|
||
from typing import TYPE_CHECKING | ||
import functools | ||
import warnings | ||
|
||
if TYPE_CHECKING: | ||
from typing import deprecated | ||
else: | ||
def deprecated(msg): | ||
def decorator(func): | ||
@functools.wraps(func) | ||
def wrapper(*args, **kwargs): | ||
warnings.warn(msg, DeprecationWarning, stacklevel=2) | ||
return func(*args, **kwargs) | ||
wrapper.__deprecated__ = msg | ||
return wrapper | ||
return decorator | ||
|
||
Deprecation of modules and attributes | ||
------------------------------------- | ||
|
||
This PEP covers deprecations of classes, functions and overloads. This | ||
allows type checkers to detect many but not all possible deprecations. | ||
To evaluate whether additional functionality would be worthwhile, I | ||
`examined <https://gist.github.com/JelleZijlstra/ff459edc5ff0918e22b56740bb28eb8b>`__ | ||
all current deprecations in the CPython standard library. | ||
|
||
I found: | ||
|
||
* 74 deprecations of functions, methods and classes (supported by this PEP) | ||
* 28 deprecations of whole modules (largely due to :pep:`594`) | ||
JelleZijlstra marked this conversation as resolved.
Show resolved
Hide resolved
|
||
* 9 deprecations of function parameters (supported by this PEP through | ||
decorating overloads) | ||
* 1 deprecation of a constant | ||
* 38 deprecations that are not easily detectable in the type system (for | ||
example, for calling :func:`asyncio.get_event_loop` without an active | ||
event loop) | ||
|
||
Modules could be marked as deprecated by adding a ``__deprecated__`` | ||
module-level constant. However, the need for this is limited, and it | ||
is relatively easy to detect usage of deprecated modules simply by | ||
grepping. Therefore, this PEP omits support for whole-module deprecations. | ||
As a workaround, users could mark all module-level classes and functions | ||
with ``@deprecated``. | ||
|
||
For deprecating module-level constants, object attributes, and function | ||
parameters, a ``Deprecated[type, message]`` type modifier, similar to | ||
``Annotated`` could be added. However, this would create a new place | ||
in the type system where strings are just strings, not forward references, | ||
complicating the implementation of type checkers. In addition, my data | ||
show that this feature is not commonly needed. | ||
|
||
Acknowledgments | ||
=============== | ||
|
||
A call with the typing-sig meetup group led to useful feedback on this | ||
proposal. | ||
|
||
Copyright | ||
========= | ||
|
||
This document is placed in the public domain or under the | ||
CC0-1.0-Universal license, whichever is more permissive. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.