Skip to content

Fail-fast on missing builtins #14550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions mypy/messages.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,6 @@
# test-data/unit/fixtures/) that provides the definition. This is used for
# generating better error messages when running mypy tests only.
SUGGESTED_TEST_FIXTURES: Final = {
"builtins.list": "list.pyi",
"builtins.dict": "dict.pyi",
"builtins.set": "set.pyi",
"builtins.tuple": "tuple.pyi",
"builtins.bool": "bool.pyi",
Expand Down
28 changes: 14 additions & 14 deletions mypy/semanal.py
Original file line number Diff line number Diff line change
Expand Up @@ -625,23 +625,23 @@ def add_implicit_module_attrs(self, file_node: MypyFile) -> None:
continue
# Need to construct the type ourselves, to avoid issues with __builtins__.list
# not being subscriptable or typing.List not getting bound
sym = self.lookup_qualified("__builtins__.list", Context())
if not sym:
continue
node = sym.node
if not isinstance(node, TypeInfo):
self.defer(node)
inst = self.named_type_or_none("builtins.list", [str_type])
if inst is None:
assert not self.final_iteration, "Cannot find builtins.list to add __path__"
self.defer()
return
typ = Instance(node, [str_type])
typ = inst
elif name == "__annotations__":
sym = self.lookup_qualified("__builtins__.dict", Context(), suppress_errors=True)
if not sym:
continue
node = sym.node
if not isinstance(node, TypeInfo):
self.defer(node)
inst = self.named_type_or_none(
"builtins.dict", [str_type, AnyType(TypeOfAny.special_form)]
)
if inst is None:
assert (
not self.final_iteration
), "Cannot find builtins.dict to add __annotations__"
self.defer()
return
typ = Instance(node, [str_type, AnyType(TypeOfAny.special_form)])
typ = inst
else:
assert t is not None, f"type should be specified for {name}"
typ = UnboundType(t)
Expand Down
8 changes: 2 additions & 6 deletions mypy/semanal_namedtuple.py
Original file line number Diff line number Diff line change
Expand Up @@ -481,13 +481,9 @@ def build_namedtuple_typeinfo(
strtype = self.api.named_type("builtins.str")
implicit_any = AnyType(TypeOfAny.special_form)
basetuple_type = self.api.named_type("builtins.tuple", [implicit_any])
dictype = self.api.named_type_or_none(
"builtins.dict", [strtype, implicit_any]
) or self.api.named_type("builtins.object")
dictype = self.api.named_type("builtins.dict", [strtype, implicit_any])
# Actual signature should return OrderedDict[str, Union[types]]
ordereddictype = self.api.named_type_or_none(
"builtins.dict", [strtype, implicit_any]
) or self.api.named_type("builtins.object")
ordereddictype = self.api.named_type("builtins.dict", [strtype, implicit_any])
fallback = self.api.named_type("builtins.tuple", [implicit_any])
# Note: actual signature should accept an invariant version of Iterable[UnionType[types]].
# but it can't be expressed. 'new' and 'len' should be callable types.
Expand Down
2 changes: 2 additions & 0 deletions test-data/unit/check-dynamic-typing.test
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ class int: pass
class type: pass
class function: pass
class str: pass
class dict: pass

[case testBinaryOperationsWithDynamicAsRightOperand]
from typing import Any
Expand Down Expand Up @@ -219,6 +220,7 @@ class int: pass
class type: pass
class function: pass
class str: pass
class dict: pass

[case testDynamicWithUnaryExpressions]
from typing import Any
Expand Down
1 change: 1 addition & 0 deletions test-data/unit/check-generics.test
Original file line number Diff line number Diff line change
Expand Up @@ -1331,6 +1331,7 @@ class type: pass
class tuple: pass
class function: pass
class str: pass
class dict: pass

[case testMultipleAssignmentWithIterable]
from typing import Iterable, TypeVar
Expand Down
8 changes: 0 additions & 8 deletions test-data/unit/check-incomplete-fixture.test
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,6 @@ import m
m.x # E: "object" has no attribute "x"
[file m.py]

[case testDictMissingFromStubs]
from typing import Dict
def f(x: Dict[int]) -> None: pass
[out]
main:1: error: Module "typing" has no attribute "Dict"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you update this test with something more exotic from typing.pyi?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try, but if having dict and list is an invariant now, maybe this test should just go away?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are other things in typing.pyi that probably shouldn't be in the default fixture, e.g. like dataclass_transform. This error is still useful there.

(In general, if we can get away with omitting something from the fixture and having devs hit this error, I think that's much more preferable than having an empty class, where degradation from missing API is less obvious. I'm fine with this PR for list and dict, because they're fundamental enough that their complete absence is annoying)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This leads me to think that maybe the "Maybe your test fixture does not define ..." message should be triggered by attribute access to a type that was defined in a fixture, not only absence of the type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In its current state, this test can be removed, and "builtins.list" and "builtins.dict" can be removed from SUGGESTED_TEST_FIXTURES, since we'd no longer have a "trip wire" for devs.

I do think this trip wire can remain useful if we warn about type missing or type's attributes missing, but
a) I can't tell how noisy that'd be
b) Implementing something like this would require determining whether a TypeInfo comes a fixture (or, perhaps, if its defn comes from a fixture). I can't see anything on either of them that allows me to know it right now. Before investing more work, I'm curious what people think.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating! And sorry my message was confusing, didn't realise there were several other testXMissingFromStubs tests.

I think logic for b) we could use is just checking if module is typing.pyi or builtins.pyi?

main:1: note: Maybe your test fixture does not define "builtins.dict"?
main:1: note: Consider adding [builtins fixtures/dict.pyi] to your test description

[case testSetMissingFromStubs]
from typing import Set
def f(x: Set[int]) -> None: pass
Expand Down
1 change: 1 addition & 0 deletions test-data/unit/check-tuples.test
Original file line number Diff line number Diff line change
Expand Up @@ -774,6 +774,7 @@ class str: pass
class bool: pass
class type: pass
class function: pass
class dict: pass


-- For loop over tuple
Expand Down
2 changes: 2 additions & 0 deletions test-data/unit/cmdline.test
Original file line number Diff line number Diff line change
Expand Up @@ -1516,6 +1516,8 @@ a.py:2: note: By default the bodies of untyped functions are not checked, consid
class object: pass
class str(object): pass
class int(object): pass
class list: pass
class dict: pass
[file dir/stdlib/sys.pyi]
[file dir/stdlib/types.pyi]
[file dir/stdlib/typing.pyi]
Expand Down
4 changes: 2 additions & 2 deletions test-data/unit/fine-grained.test
Original file line number Diff line number Diff line change
Expand Up @@ -1809,8 +1809,8 @@ def f() -> Iterator[None]:
[typing fixtures/typing-medium.pyi]
[builtins fixtures/list.pyi]
[triggered]
2: <b>, __main__
3: <b>, __main__, a
2: <b>, <b[wildcard]>, __main__
3: <b>, <b[wildcard]>, __main__, a
[out]
main:2: note: Revealed type is "contextlib.GeneratorContextManager[None]"
==
Expand Down
1 change: 1 addition & 0 deletions test-data/unit/fixtures/__init_subclass__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ class int: pass
class bool: pass
class str: pass
class function: pass
class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/__new__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ class int: pass
class bool: pass
class str: pass
class function: pass
class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/alias.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,5 @@ class str: pass
class function: pass

bytes = str

class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/any.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,5 @@ class int: pass
class str: pass

def any(i: Iterable[T]) -> bool: pass

class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/attr.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,5 @@ class complex:
class str: pass
class ellipsis: pass
class tuple: pass
class list: pass
class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/bool.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,4 @@ class str: pass
class ellipsis: pass
class list(Generic[T]): pass
class property: pass
class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/callable.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,4 @@ class str:
def __eq__(self, other: 'str') -> bool: pass
class ellipsis: pass
class list: ...
class dict: pass
3 changes: 3 additions & 0 deletions test-data/unit/fixtures/classmethod.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,6 @@ class bool: pass
class ellipsis: pass

class tuple(typing.Generic[_T]): pass

class list: pass
class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/complex.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ class int: pass
class float: pass
class complex: pass
class str: pass
class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/complex_tuple.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ class float: pass
class complex: pass
class str: pass
class ellipsis: pass
class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/divmod.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,5 @@ class ellipsis: pass

_N = TypeVar('_N', int, float)
def divmod(_x: _N, _y: _N) -> Tuple[_N, _N]: ...

class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/exception.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ class object:
class type: pass
class tuple(Generic[T]):
def __ge__(self, other: object) -> bool: ...
class list: pass
class dict: pass
class function: pass
class int: pass
class str: pass
Expand Down
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/f_string.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,5 @@ class str:
def format(self, *args) -> str: pass
def join(self, l: List[str]) -> str: pass


class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/fine_grained.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,4 @@ class tuple(Generic[T]): pass
class function: pass
class ellipsis: pass
class list(Generic[T]): pass
class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/float.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,5 @@ class float:
def __int__(self) -> int: ...
def __mul__(self, x: float) -> float: ...
def __rmul__(self, x: float) -> float: ...

class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/for.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,4 @@ class str: pass # for convenience

class list(Iterable[t], Generic[t]):
def __iter__(self) -> Iterator[t]: pass
class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/function.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ class type: pass
class function: pass
class int: pass
class str: pass
class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/isinstance.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,5 @@ class str:
class ellipsis: pass

NotImplemented = cast(Any, None)

class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/isinstance_python3_10.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,5 @@ class str:
class ellipsis: pass

NotImplemented = cast(Any, None)

class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/list.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,5 @@ class str:
class bool(int): pass

property = object() # Dummy definition.

class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/module_all.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ class list(Generic[_T], Sequence[_T]):
def __add__(self, rhs: Sequence[_T]) -> list[_T]: pass
class tuple(Generic[_T]): pass
class ellipsis: pass
class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/notimplemented.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ class bool: pass
class int: pass
class str: pass
NotImplemented = cast(Any, None)
class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/object_hashable.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ class float: ...
class str: ...
class ellipsis: ...
class tuple: ...
class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/ops.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,5 @@ def __print(a1: object = None, a2: object = None, a3: object = None,
a4: object = None) -> None: pass

class ellipsis: pass

class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/property.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ class function: pass
property = object() # Dummy definition
class classmethod: pass

class list: pass
class dict: pass
class int: pass
class str: pass
Expand Down
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/set.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,5 @@ class set(Iterable[T], Generic[T]):
def add(self, x: T) -> None: pass
def discard(self, x: T) -> None: pass
def update(self, x: Set[T]) -> None: pass

class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/slice.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@ class str: pass

class slice: pass
class ellipsis: pass
class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/staticmethod.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,4 @@ class int:
class str: pass
class bytes: pass
class ellipsis: pass
class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/transform.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,5 @@ def __print(a1=None, a2=None, a3=None, a4=None):
# Do not use *args since this would require list and break many test
# cases.
pass

class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/tuple-simple.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,4 @@ class function: pass
# We need int for indexing tuples.
class int: pass
class str: pass # For convenience
class dict: pass
2 changes: 2 additions & 0 deletions test-data/unit/fixtures/tuple.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,5 @@ def isinstance(x: object, t: type) -> bool: pass
def sum(iterable: Iterable[T], start: Optional[T] = None) -> T: pass

class BaseException: pass

class dict: pass
1 change: 1 addition & 0 deletions test-data/unit/fixtures/union.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ class tuple(Generic[T]): pass
# We need int for indexing tuples.
class int: pass
class str: pass # For convenience
class dict: pass
9 changes: 7 additions & 2 deletions test-data/unit/lib-stub/builtins.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,13 @@ class function:
__name__: str
class ellipsis: pass

from typing import Generic, Sequence, TypeVar
from typing import Generic, Iterator, Sequence, TypeVar
_T = TypeVar('_T')
class list(Generic[_T], Sequence[_T]): pass
class list(Generic[_T], Sequence[_T]):
def __contains__(self, item: object) -> bool: pass
def __getitem__(self, key: int) -> _T: pass
def __iter__(self) -> Iterator[_T]: pass

class dict: pass

# Definition of None is implicit
Loading