Skip to content

Commit 1690984

Browse files
committed
Stop TypedDictAnalyzer from leaking synthetic types
Fixes #10007 Mypy currently crashes when you try: 1. Creating a class-based TypedDict containing a malformed type hint 2. Asking it to compute fine-grained dependencies, either via running dmypy or by setting the `--cache-fine-grained` flag. Here is the exact sequence of events that leads to this crash: 1. Since the type annotation is malformed, semanal initially gives the type annotation a type of `RawExpressionType`. 2. TypedDictAnalyzer (correctly) determines determines that the type of the malformed type annotation should be treated as just `Any` in: https://github.com/python/mypy/blob/f5ce4ee6ca7e2d8bb1cde8a2b49865f53bbacff5/mypy/semanal_typeddict.py#L289 3. TypedDictAnalyzer forgets to modify `stmt.type` like we normally do after calling `self.anal_type` in normal classes: https://github.com/python/mypy/blob/f5ce4ee6ca7e2d8bb1cde8a2b49865f53bbacff5/mypy/semanal.py#L3022 4. Mypy _does_ use the `Any` type when constructing the TypeInfo for the TypedDict. This is why mypy will not crash under most conditions: the correct type is being used in most places. 5. Setting `--cache-fine-grained` will make mypy perform one final pass against the AST to compute fine-grained dependencies. As a part of this process, it traverses the AssigmentStatement's `type` field using TypeTriggersVisitor. 6. TypeTriggersVisitor is _not_ a SyntheticTypeVisitor. So, the visitor trips an assert when we try traversing into the RawExpressionType. Interestingly, this same crash does not occur for NamedTuples despite the fact that NamedTupleAnalyzer also does not set `stmt.type`: https://github.com/python/mypy/blob/f5ce4ee6ca7e2d8bb1cde8a2b49865f53bbacff5/mypy/semanal_namedtuple.py#L177 It turns out this is because semanal.py will end up calling the `analyze_class_body_common(...)` function after NamedTupleAnalyzer runs, but _not_ after TypedDictAnalyzer runs: - https://github.com/python/mypy/blob/f5ce4ee6ca7e2d8bb1cde8a2b49865f53bbacff5/mypy/semanal.py#L1510 - https://github.com/python/mypy/blob/f5ce4ee6ca7e2d8bb1cde8a2b49865f53bbacff5/mypy/semanal.py#L1479 I'm not sure why this is: ideally, the two analyzers ought to have as similar semantics as possible. But refactoring this felt potentially disruptive, so I went for the narrower route of just patching TypedDictAnalyzer.
1 parent a3a5d73 commit 1690984

File tree

3 files changed

+37
-1
lines changed

3 files changed

+37
-1
lines changed

mypy/semanal_typeddict.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -294,6 +294,12 @@ def analyze_typeddict_classdef_fields(
294294
)
295295
if analyzed is None:
296296
return None, [], set() # Need to defer
297+
# TypedDictAnalyzer sets the AssignmentStmt type here, but
298+
# NamedTupleAnalyzer doesn't and instead has semanal.py set it
299+
# by calling analyze_class_body_common after.
300+
#
301+
# TODO: Resolve this inconsistency?
302+
stmt.type = analyzed
297303
types.append(analyzed)
298304
# ...despite possible minor failures that allow further analyzis.
299305
if stmt.type is None or hasattr(stmt, "new_syntax") and not stmt.new_syntax:

test-data/unit/check-semanal-error.test

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,3 +152,33 @@ class C:
152152
x: P[int] = C()
153153
[builtins fixtures/tuple.pyi]
154154
[out]
155+
156+
[case testSemanalDoesNotLeakSyntheticTypes]
157+
# flags: --cache-fine-grained
158+
from typing import Generic, NamedTuple, TypedDict, TypeVar
159+
from dataclasses import dataclass
160+
161+
T = TypeVar('T')
162+
class Wrap(Generic[T]): pass
163+
164+
invalid_1: 1 + 2 # E: Invalid type comment or annotation
165+
invalid_2: Wrap[1 + 2] # E: Invalid type comment or annotation
166+
167+
class A:
168+
invalid_1: 1 + 2 # E: Invalid type comment or annotation
169+
invalid_2: Wrap[1 + 2] # E: Invalid type comment or annotation
170+
171+
class B(NamedTuple):
172+
invalid_1: 1 + 2 # E: Invalid type comment or annotation
173+
invalid_2: Wrap[1 + 2] # E: Invalid type comment or annotation
174+
175+
class C(TypedDict):
176+
invalid_1: 1 + 2 # E: Invalid type comment or annotation
177+
invalid_2: Wrap[1 + 2] # E: Invalid type comment or annotation
178+
179+
@dataclass
180+
class D:
181+
invalid_1: 1 + 2 # E: Invalid type comment or annotation
182+
invalid_2: Wrap[1 + 2] # E: Invalid type comment or annotation
183+
[builtins fixtures/dict.pyi]
184+
[typing fixtures/typing-typeddict.pyi]

test-data/unit/semanal-typeddict.test

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,4 +42,4 @@ MypyFile:1(
4242
NameExpr(x)
4343
TempNode:4(
4444
Any)
45-
str?)))
45+
builtins.str)))

0 commit comments

Comments
 (0)