Skip to content

Replace redundant sections with link to the typing documentation #7594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

Replace redundant sections with link to the typing documentation #7594

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
249b555
Replace redundant sections with link to the typing documentation
srittau Apr 5, 2022
42fee79
Merge branch 'master' into link-to-typing-docs
srittau May 21, 2022
f915ec0
Remove the link to PEP 484, we link to the more comprehensive stubs d…
srittau May 21, 2022
e25542b
Link to the feature tracker issues
srittau May 21, 2022
1ac4b3d
Merge branch 'master' into link-to-typing-docs
AlexWaygood May 31, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
265 changes: 9 additions & 256 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@ recommend starting by opening an issue laying out what you want to do.
That lets a conversation happen early in case other contributors disagree
with what you'd like to do or have ideas that will help you do it.

### Format
### What are stub files?

Each Python module is represented by a `.pyi` "stub file". This is a
syntactically valid Python file, although it usually cannot be run by
Expand All @@ -220,10 +220,6 @@ the methods are empty.
Python function annotations ([PEP 3107](https://www.python.org/dev/peps/pep-3107/))
are used to describe the signature of each function or method.

See [PEP 484](http://www.python.org/dev/peps/pep-0484/) for the exact
syntax of the stub files and [below](#stub-file-coding-style) for the
coding style used in typeshed.

### Auto-generating stub files

Typeshed includes `scripts/create_baseline_stubs.py`.
Expand All @@ -245,259 +241,16 @@ When the script has finished running, it will print instructions telling you wha
If it has been a while since you set up the virtualenv, make sure you have
the latest mypy (`pip install -r requirements-tests.txt`) before running the script.

### Supported type system features

Since PEP 484 was accepted, there have been many other PEPs that added
new features to the Python type system. In general, new features can
be used in typeshed as soon as the PEP has been accepted and implemented
and most type checkers support the new feature.

Accepted features that *cannot* yet be used in typeshed include:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this list is still useful, and it makes sense to keep it in typeshed because it's specifically about this repo's policies.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops, I kind of forgot about this PR.

Maybe it's best to link to https://github.com/python/typeshed/issues?q=is%3Aissue+is%3Aopen+label%3Afeature-tracker, instead of listing the missing features? The issues are more likely to be kept up-to-date than the list in CONTRIBUTING.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, sounds good

Copy link
Member

@AlexWaygood AlexWaygood May 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also consider using a meta-issue tracker to track the issue trackers. The advantage of using an issue to track the issue trackers is that items will automatically be ticked off as completed if issue trackers on the bullet-pointed list are closed. We could link to the meta-issue tracker from CONTRIBUTING.md and pin it on the issue tracker.

- [PEP 570](https://www.python.org/dev/peps/pep-0570/) (positional-only
arguments): see [#4972](https://github.com/python/typeshed/issues/4972),
use argument names prefixed with `__` instead
- [PEP 613](https://www.python.org/dev/peps/pep-0613/) (TypeAlias):
see [#4913](https://github.com/python/typeshed/issues/4913)

The following features are partially supported:
- [PEP 585](https://www.python.org/dev/peps/pep-0585/) (builtin
generics): see [#4820](https://github.com/python/typeshed/issues/4820),
mostly supported but bugs remain for a few specific cases
- [PEP 612](https://www.python.org/dev/peps/pep-0612/) (ParamSpec):
see [#4827](https://github.com/python/typeshed/issues/4827),
supported in some contexts but requires `# type: ignore` comments

Supported features include:
- [PEP 544](https://www.python.org/dev/peps/pep-0544/) (Protocol)
- [PEP 586](https://www.python.org/dev/peps/pep-0586/) (Literal)
- [PEP 591](https://www.python.org/dev/peps/pep-0591/) (Final/@final)
- [PEP 589](https://www.python.org/dev/peps/pep-0589/) (TypedDict)
- [PEP 604](https://www.python.org/dev/peps/pep-0604/) (`Foo | Bar` union syntax)
- [PEP 647](https://www.python.org/dev/peps/pep-0647/) (TypeGuard):
see [#5406](https://github.com/python/typeshed/issues/5406)

Features from the `typing` module that are not present in all
supported Python 3 versions must be imported from `typing_extensions`
instead in typeshed stubs. This currently affects:

- `Final` and `@final` (new in Python 3.8)
- `Literal` (new in Python 3.8)
- `SupportsIndex` (new in Python 3.8)
- `TypedDict` (new in Python 3.8)
- `Concatenate` (new in Python 3.10)
- `ParamSpec` (new in Python 3.10)
- `TypeGuard` (new in Python 3.10)

Two exceptions are `Protocol` and `runtime_checkable`: although
these were added in Python 3.8, they can be used in stubs regardless
of Python version.

### What to include

Stubs should include the complete interface (classes, functions,
constants, etc.) of the module they cover, but it is not always
clear exactly what is part of the interface.

The following should always be included:
- All objects listed in the module's documentation.
- All objects included in ``__all__`` (if present).

Other objects may be included if they are being used in practice
or if they are not prefixed with an underscore. This means
that typeshed will generally accept contributions that add missing
objects, even if they are undocumented. Undocumented objects should
be marked with a comment of the form ``# undocumented``.
Example:

```python
def list2cmdline(seq: Sequence[str]) -> str: ... # undocumented
```

We accept such undocumented objects because omitting objects can confuse
users. Users who see an error like "module X has no attribute Y" will
not know whether the error appeared because their code had a bug or
because the stub is wrong. Although it may also be helpful for a type
checker to point out usage of private objects, we usually prefer false
negatives (no errors for wrong code) over false positives (type errors
for correct code). In addition, even for private objects a type checker
can be helpful in pointing out that an incorrect type was used.

### What to do when a project's documentation and implementation disagree

Type stubs are meant to be external type annotations for a given
library. While they are useful documentation in its own merit, they
augment the project's concrete implementation, not the project's
documentation. Whenever you find them disagreeing, model the type
information after the actual implementation and file an issue on the
project's tracker to fix their documentation.

### Stub versioning

You can use checks
like `if sys.version_info >= (3, 8):` to denote new functionality introduced
in a given Python version or solve type differences. When doing so, only use
one-tuples or two-tuples. Because of this, if a given functionality was
introduced in, say, Python 3.7.4, your check:

* should be expressed as `if sys.version_info >= (3, 7):`
* should NOT be expressed as `if sys.version_info >= (3, 7, 4):`
* should NOT be expressed as `if sys.version_info >= (3, 8):`

When your stub contains if statements for different Python versions,
always put the code for the most recent Python version first.

### Incomplete stubs

We accept partial stubs, especially for larger packages. These need to
follow the following guidelines:

* Included functions and methods must list all arguments, but the arguments
can be left unannotated. Do not use `Any` to mark unannotated arguments
or return values.
* Partial classes must include a `__getattr__()` method marked with an
`# incomplete` comment (see example below).
* Partial modules (i.e. modules that are missing some or all classes,
functions, or attributes) must include a top-level `__getattr__()`
function marked with an `# incomplete` comment (see example below).
* Partial packages (i.e. packages that are missing one or more sub-modules)
must have a `__init__.pyi` stub that is marked as incomplete (see above).
A better alternative is to create empty stubs for all sub-modules and
mark them as incomplete individually.

Example of a partial module with a partial class `Foo` and a partially
annotated function `bar()`:

```python
def __getattr__(name: str) -> Any: ... # incomplete

class Foo:
def __getattr__(self, name: str) -> Any: ... # incomplete
x: int
y: str

def bar(x: str, y, *, z=...): ...
```

## Stub file coding style

### Syntax example

The below is an excerpt from the types for the `datetime` module.

```python
MAXYEAR: int
MINYEAR: int

class date:
def __new__(cls: Type[_S], year: int, month: int, day: int) -> _S: ...
@classmethod
def fromtimestamp(cls: Type[_S], __timestamp: float) -> _S: ...
@classmethod
def today(cls: Type[_S]) -> _S: ...
@classmethod
def fromordinal(cls: Type[_S], __n: int) -> _S: ...
@property
def year(self) -> int: ...
def replace(self, year: int = ..., month: int = ..., day: int = ...) -> date: ...
def ctime(self) -> str: ...
def weekday(self) -> int: ...
```
### Stub Content

### Conventions

Stub files are *like* Python files and you should generally expect them
to look the same. Your tools should be able to successfully treat them
as regular Python files. However, there are a few important differences
you should know about.

Style conventions for stub files are different from PEP 8. The general
rule is that they should be as concise as possible. Specifically:
* all function bodies should be empty;
* prefer ``...`` over ``pass``;
* prefer ``...`` on the same line as the class/function signature;
* avoid vertical whitespace between consecutive module-level functions,
names, or methods and fields within a single class;
* use a single blank line between top-level class definitions, or none
if the classes are very small;
* do not use docstrings;
* use variable annotations instead of type comments, even for stubs
that target older versions of Python.

Stub files should only contain information necessary for the type
checker, and leave out unnecessary detail:
* for arguments with a default, use `...` instead of the actual
default;
* for arguments that default to `None`, use `Foo | None` explicitly
(see below for details);
* use `float` instead of `int | float`.

Some further tips for good type hints:
* use built-in generics (`list`, `dict`, `tuple`, `set`), instead
of importing them from `typing`, **except** in type aliases, in base classes, and for
arbitrary length tuples (`Tuple[int, ...]`);
* use `X | Y` instead of `Union[X, Y]` and `X | None`, instead of
`Optional[X]`, **except** when it is not possible due to mypy bugs (type aliases and base classes);
* in Python 3 stubs, import collections (`Mapping`, `Iterable`, etc.)
from `collections.abc` instead of `typing`;
* avoid invariant collection types (`list`, `dict`) in argument
positions, in favor of covariant types like `Mapping` or `Sequence`;
* avoid union return types: https://github.com/python/mypy/issues/1693;
* in Python 2, whenever possible, use `unicode` if that's the only
possible type, and `Text` if it can be either `unicode` or `bytes`;
* use platform checks like `if sys.platform == 'win32'` to denote
platform-dependent APIs;
* use mypy error codes for mypy-specific `# type: ignore` annotations,
e.g. `# type: ignore[override]` for Liskov Substitution Principle violations.

Imports in stubs are considered private (not part of the exported API)
unless:
* they use the form ``from library import name as name`` (sic, using
explicit ``as`` even if the name stays the same); or
* they use the form ``from library import *`` which means all names
from that library are exported.

When adding type hints, avoid using the `Any` type when possible. Reserve
the use of `Any` for when:
* the correct type cannot be expressed in the current type system; and
* to avoid union returns (see above).

Note that `Any` is not the correct type to use if you want to indicate
that some function can accept literally anything: in those cases use
`object` instead.

Stub files support forward references natively. In other words, the
order of class declarations and type aliases does not matter in
a stub file. You can also use the name of the class within its own
body. Focus on making your stubs clear to the reader. Avoid using
string literals in type annotations.

Type variables and aliases you introduce purely for legibility reasons
should be prefixed with an underscore to make it obvious to the reader
they are not part of the stubbed API.

When adding type annotations for context manager classes, annotate
the return type of `__exit__` as bool only if the context manager
sometimes suppresses exceptions -- if it sometimes returns `True`
at runtime. If the context manager never suppresses exceptions,
have the return type be either `None` or `bool | None`. If you
are not sure whether exceptions are suppressed or not or if the
context manager is meant to be subclassed, pick `bool | None`.
See https://github.com/python/mypy/issues/7214 for more details.

`__enter__` methods and other methods that return instances of the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The typing docs don't mention this

current class should be annotated with the `_typeshed.Self` type
variable ([example](https://github.com/python/typeshed/pull/5698)).

A few guidelines for protocol names below. In cases that don't fall
into any of those categories, use your best judgement.

* Use plain names for protocols that represent a clear concept
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't in the linked doc

(e.g. `Iterator`, `Container`).
* Use `SupportsX` for protocols that provide callable methods (e.g.
`SupportsInt`, `SupportsRead`, `SupportsReadSeek`).
* Use `HasX` for protocols that have readable and/or writable attributes
or getter/setter methods (e.g. `HasItems`, `HasFileno`).
Please see [the typing documentation](https://typing.readthedocs.io/en/latest/source/stubs.html)
for type system features supported in stubs, best practices, and our
style guide.

As typing is constantly evolving, not all type checkers support all features
that would be useful in stub files. See our
["feature-tracker" issue label](/python/typeshed/issues?q=is%3Aissue+is%3Aopen+label%3Afeature-tracker)
for details on which features can't be used at the moment.

## Submitting Changes

Expand Down