PEP 655: Support Required[] inside TypedDict

[davidfstr]

Description

Adds support for the Required[] syntax inside TypedDicts as specified by PEP 655. Note that the NotRequired[] syntax from that PEP is not in this initial draft, because I want to see first if there is any feedback on how I'm implementing Required[].

A separate PR will be made to the typing_extensions module to enable runtime support for Required[].

CC sponsor @gvanrossum

Test Plan

New tests exist in test-data/unit/check-typeddict.test after the -- Required[] line, and can be run individually with lines like:

• pytest -n0 mypy/test/testcheck.py::TypeCheckSuite::testRequiredOnlyAllowsOneItem

[davidfstr]

Looks like I have a CI failure to investigate...

[davidfstr]

CI has passed. Ready for folks to take a look.

[davidfstr]

Outline of changes, to hopefully make the review easier:

• There are two kinds of TypedDict declarations mypy recognizes, a class-based form and an assignment-based form:

class Movie(TypedDict, total=False):  # class-based
    title: Required[str]
    year: int

Movie = TypedDict('Movie', {  # assignment-based
    'title': Required[str],
    'year': int,
}, total=False)

• Those declarations are parsed by, respectively:
  • TypedDictAnalyzer.analyze_typeddict_classdef @ mypy/semanal_typeddict.py
  • SemanticAnalyzer.analyze_typeddict_assign @ mypy/semanal.py
• This commit alters both of these functions to recognize Required[...] as valid wrapper around each key's type, and to treat such-marked keys as required.
• Both of these functions obtain a Type for each key in the TypedDict by calling SemanticAnalyser.anal_type() on the key type.
• anal_type() is changed by this commit to recognize Required[...] as a RequiredType in:
  • try_analyze_special_unbound_type @ mypy/typeanal.py
• We only want to allow the Required[...] syntax in the very narrow situation of a TypedDict key definition. So the top call of SemanticAnalyser.anal_type() takes an allow_required=True parameter that can be used to enable recogition of Required[...] by the two TypedDict-parsing methods, but to disable recognition by default in other contexts.

[JukkaL]

I did a quick review pass and overall looks good. It's okay to move forward with supporting NotRequired as an extension.

[davidfstr]

Thanks for the review @JukkaL !

I'll convert this PR to draft for now and add additional commits for the NotRequired case. Separately, I'll make a PR for the typing_extensions module.

[davidfstr]

PR is extended with support for NotRequired[] and is ready for another round of review.

[JukkaL]

Would you be able to fix the merge conflict -- I can review this now. Apologies for the very slow response!

[davidfstr]

I’ve been disconnected from open source for several months to take care of some life issues. I’m hoping I’ll have enough energy to reengage here in a month or so.

In the meantime I welcome any rebase/deconflict support from any observers who’d like this branch to be merged faster.

[JukkaL]

No problem!

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

steam.py (https://github.com/Gobot1234/steam.py)
- steam/iterators.py:70: error: Name "Required" is not defined  [name-defined]
- steam/iterators.py:71: error: Name "Required" is not defined  [name-defined]
- steam/trade.py:70: error: Variable "typing_extensions.Required" is not valid as a type  [valid-type]
- steam/trade.py:70: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
- steam/trade.py:71: error: Variable "typing_extensions.Required" is not valid as a type  [valid-type]
- steam/trade.py:71: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
- steam/trade.py:275: error: Argument 1 to "update" of "TypedDict" has incompatible type "AssetDict"; expected "TypedDict({'instanceid'?: Required?[builtins.str], 'classid'?: Required?[builtins.str], 'market_name'?: str, 'currency'?: int, 'name'?: str, 'market_hash_name'?: str, 'name_color'?: str, 'background_color'?: str, 'type'?: str, 'descriptions'?: Dict[str, str], 'market_actions'?: List[Dict[str, str]], 'tags'?: List[Dict[str, str]], 'actions'?: List[Dict[str, str]], 'icon_url'?: str, 'icon_url_large'?: str, 'tradable'?: bool, 'marketable'?: bool, 'commodity'?: int, 'fraudwarnings'?: List[str]})"  [typeddict-item]
+ steam/trade.py:275: error: Argument 1 to "update" of "TypedDict" has incompatible type "AssetDict"; expected "TypedDict({'instanceid'?: str, 'classid'?: str, 'market_name'?: str, 'currency'?: int, 'name'?: str, 'market_hash_name'?: str, 'name_color'?: str, 'background_color'?: str, 'type'?: str, 'descriptions'?: Dict[str, str], 'market_actions'?: List[Dict[str, str]], 'tags'?: List[Dict[str, str]], 'actions'?: List[Dict[str, str]], 'icon_url'?: str, 'icon_url_large'?: str, 'tradable'?: bool, 'marketable'?: bool, 'commodity'?: int, 'fraudwarnings'?: List[str]})"  [typeddict-item]
- steam/http.py:70: error: Name "Required" is not defined  [name-defined]
- steam/http.py:71: error: Name "Required" is not defined  [name-defined]
- steam/state.py:70: error: Name "Required" is not defined  [name-defined]
- steam/state.py:71: error: Name "Required" is not defined  [name-defined]

[JukkaL]

Looks good now. Thank you @davidfstr and @97littleleaf11! This has been a frequently requested feature.

[97littleleaf11]

@JukkaL Thanks for your review! And this PR also looks good to me.

[davidfstr]

Thanks @97littleleaf11 for getting this branch finished while I've had to step back for a while! And thank you @JukkaL for getting it reviewed. Team work :)