bpo-40979: refactored typing.rst; same content, new sub-sections and ordering

[ramalho]

Here is how I reorganized the content previously in the "Classes, functions, and decorators" section:

# Special typing primitives	
	Annotated
	Any
	Callable
	ClassVar
	Final
	ForwardRef
	Generic
	Literal
	NewType
	NoReturn
	Optional
	Protocol
	Type
	TypedDict
	TypeVar
	Union

# Generic concrete collections	
	ChainMap
	Counter
	DefaultDict
	Deque
	Dict
	FrozenSet
	List
	NamedTuple
	OrderedDict
	Set
	Tuple

# Generic ABCs	
	AbstractSet
	AsyncContextManager
	AsyncGenerator
	AsyncIterable
	AsyncIterator
	Awaitable
	ByteString
	Collection
	Container
	ContextManager
	Coroutine
	Generator
	Hashable
	io.IO
	io.BytesIO
	io.TextIO
	ItemsView
	Iterable
	Iterator
	KeysView
	Mapping
	MappingView
	MutableMapping
	MutableSequence
	MutableSet
	re.Pattern
	re.Match
	Sequence
	Sized
	ValuesView	

# Protocols	
	Reversible
	SupportsAbs
	SupportsBytes
	SupportsComplex
	SupportsFloat
	SupportsIndex
	SupportsInt
	SupportsRound	

# Aliases and constants	
	AnyStr
	Text
	TYPE_CHECKING

# Functions and decorators	
	cast
	final
	get_args
	get_origin
	get_type_hints
	no_type_check
	no_type_check_decorator
	overload
	runtime_checkable
	type_check_only

https://bugs.python.org/issue40979

[gvanrossum]

Is this still in progress? You seem to have many commits and the tests still don't pass.

Also, I'm not so keen on alphabetizing entries. I'd rather order them from most to least important. @rhettinger Do you agree?

Finally the io and re sub-namespaces were basically a mistake. We need to document them until we decide to deprecate them, but IMO the few objects inside them should primarily be documented at the top level.

[ramalho]

Thanks, @gvanrossum.

Is this still in progress?

I am done with the PR, unless changes are requested.

You seem to have many commits...

I made one huge commit with all changes, then a friend suggested it would be hard to review, so for the benefit of reviewers I made several commits working section by section. This makes it easy to verify that no content went missing during the reorganization.

and the tests still don't pass.

The last failing check was the lack of a blurb, which I added about 10 hours ago, and then I saw all checks passing. I did make some rookie mistakes before, but they are all fixed. Please let me know which tests are not passing now.

Also, I'm not so keen on alphabetizing entries. I'd rather order them from most to least important. @rhettinger Do you agree?

I used "importance" as the main criterion for ordering the sections. Within each section, I believe the user is better served with an alphabetical listing when the section is longer than a screenful. I wouldn't know how to rank the entries in Generic ABCs by importance. From an editorial perspective, orderings that are not clear to the reader are not helpful.

Finally the io and re sub-namespaces were basically a mistake. We need to document them until we decide to deprecate them, but IMO the few objects inside them should primarily be documented at the top level.

The entries for (IO, BytesIO, TextIO) and (Pattern, Match) appear together in the .rst file, but without the sub-namespace prefix. I only added a sentence in each entry about the sub-namespace, which we can simply remove when necessary.

[gvanrossum]

I used "importance" as the main criterion for ordering the sections. Within each section, I believe the user is better served with an alphabetical listing when the section is longer than a screenful. I wouldn't know how to rank the entries in Generic ABCs by importance. From an editorial perspective, orderings that are not clear to the reader are not helpful.

Hm; I sometimes like to read documentation starting from the top (since where else are you going to start? :-) until it gets boring, and then it's nice to have it be in some useful order. Alphabetical order may break a tie, but it was invented in order to make manual search easier. Since we have search now, that no longer applies, and we can do something more useful. (How do you organize things in your books? Hopefully not alphabetical.) If you really think it ought to be alphabetical per section, maybe we can introduce sub-sections. For example, here's a possible way to split the first two sections in parts:

# Special typing primitives

## Special types

These can be used as types in annotations and don't use `[]`.

	Any
	NoReturn

## Special forms

These can be used as types in annotations using `[]`, each having a unique syntax.

	Annotated
	Callable
	ClassVar
	Final
	Literal
	Optional
	Tuple  (Moved here! It's a special form.)
	Type
	Union

## Other special directives

These are not used in annotations.

	ForwardRef
	Generic
	NewType
	Protocol
	TypedDict
	TypeVar

# Generic concrete collections

## Corresponding to built-in types

	Dict
	FrozenSet
	List
	Set

## Corresponding to types in `collections`

	ChainMap
	Counter
	DefaultDict
	Deque
	NamedTuple
	OrderedDict

I have issues with the "generic ABCs" too: some of them are not generic, some are not abstract, and probably these should be separated by origin module, i.e. abc, collections.abc, and other modules.

I assume BytesIO is a typo for BinaryIO.

I would also separate functions and decorators into two sub-sections, and honestly there's no reason to put TYPE_CHECKING in the same section as AnyStr and Text.

[rhettinger]

Also, I'm not so keen on alphabetizing entries. I'd rather order them from most to least important. @rhettinger Do you agree?

Yes, that makes sense.

[ramalho]

Great proposal, @gvanrossum, thanks. I will follow it, but I have a couple of questions:

1. Why is Tuple a special form from the user's perspective? As a user, I don't see it as more special than Dict. Is the rationale for treating it as a special form related to its implementation?

2. I like the Other special directives grouping and the observation "These are not used in annotations." May we add: "They are building blocks for declaring types." ?

I have issues with the "generic ABCs" too: some of them are not generic, some are not abstract, and probably these should be separated by origin module, i.e. abc, collections.abc, and other modules.

Agreed, I will make a proposal to split that section.

I assume BytesIO is a typo for BinaryIO.

Yes.

I would also separate functions and decorators into two sub-sections, and honestly there's no reason to put TYPE_CHECKING in the same section as AnyStr and Text.

TYPE_CHECKING is the prototypical one-of. Maybe it needs a section of its own, perhaps in the top ⅓ of typing.rst, above the existing Classes, functions, and decorators section. The same section could mention the typing.type_check_only decorator, which is related.

[gvanrossum]

1. Why is Tuple a special form from the user's perspective? As a user, I don't see it as more special than Dict. Is the rationale for treating it as a special form related to its implementation?

No, it's special because it is variadic. You cannot create a type like that using inheritance from Generic. Also because it supports the special form Tuple[t1, ...] (with literal ... meaning ellipsis) which again cannot be done by regular generic types.

2. I like the Other special directives grouping and the observation "These are not used in annotations." May we add: "They are building blocks for declaring types." ?

Of course. I did not mean to prescribe what you could write in the introduction of each section or subsection -- I only meant to clarify my intention for the grouping to you, so you can write something.

I would also separate functions and decorators into two sub-sections, and honestly there's no reason to put TYPE_CHECKING in the same section as AnyStr and Text.

TYPE_CHECKING is the prototypical one-of. Maybe it needs a section of its own, perhaps in the top ⅓ of typing.rst, above the existing Classes, functions, and decorators section. The same section could mention the typing.type_check_only decorator, which is related.

I am fine with a section on its own for TYPE_CHECKING -- it should just go in a section on "Constants". Many other modules have that in their docs I believe.

I am a little hesitant to document type_check_only since it does not exist at runtime, but I could not find any other docs for it except PEP 484 itself, so I guess it's useful. There are other reasons to use if TYPE_CHECKING though -- in mypy it's mostly used for imports that at runtime are delayed until they are needed for some advanced functionality, but which are needed by the type checker to be able to type-check everything (e.g. here).

[gvanrossum]

Are you going to do the proposed reordering?

[gvanrossum]

LGTM! I think I'll merge it now.

[bedevere-bot]

@gvanrossum: Please replace # with GH- in the commit message next time. Thanks!

[miss-islington]

Thanks @ramalho for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

[miss-islington]

Thanks @ramalho for the PR, and @gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

[miss-islington]

Sorry, @ramalho and @gvanrossum, I could not cleanly backport this to 3.8 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker ab72fdeb82c3ab045b480cd4bb4f928c12653ecb 3.8

[miss-islington]

Sorry @ramalho and @gvanrossum, I had trouble checking out the 3.9 backport branch.
Please backport using cherry_picker on command line.
cherry_picker ab72fdeb82c3ab045b480cd4bb4f928c12653ecb 3.9

[bedevere-bot]

GH-21843 is a backport of this pull request to the 3.9 branch.