Skip to content

Option to disallow explicit Any types (--disallow-any=explicit) #3579

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Jul 1, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/source/command_line.rst
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ Here are some more useful flags:

- ``--disallow-any`` disallows various types of ``Any`` in a module.
The option takes a comma-separated list of the following values:
``unimported``, ``unannotated``, ``expr``, ``decorated``.
``unimported``, ``unannotated``, ``expr``, ``decorated``, ``explicit``.

``unimported`` disallows usage of types that come from unfollowed imports
(such types become aliases for ``Any``). Unfollowed imports occur either
Expand All @@ -301,6 +301,8 @@ Here are some more useful flags:
``decorated`` disallows functions that have ``Any`` in their signature
after decorator transformation.

``explicit`` disallows explicit ``Any`` in type positions such as type
annotations and generic type parameters.

- ``--disallow-untyped-defs`` reports an error whenever it encounters
a function definition without type annotations.
Expand Down
4 changes: 2 additions & 2 deletions docs/source/config_file.rst
Original file line number Diff line number Diff line change
Expand Up @@ -150,8 +150,8 @@ overridden by the pattern sections matching the module name.
- ``disallow_any`` (Comma-separated list, default empty) is an option to
disallow various types of ``Any`` in a module. The flag takes a
comma-separated list of the following arguments: ``unimported``,
``unannotated``, ``expr``. For explanations see the discussion for the
:ref:`--disallow-any <disallow-any>` option.
``unannotated``, ``expr``, ``decorated``, ``explicit``. For explanations
see the discussion for the :ref:`--disallow-any <disallow-any>` option.

- ``disallow_untyped_calls`` (Boolean, default False) disallows
calling functions without type annotations from functions with type
Expand Down
5 changes: 4 additions & 1 deletion mypy/checker.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
ARG_POS, MDEF,
CONTRAVARIANT, COVARIANT)
from mypy import nodes
from mypy.typeanal import has_any_from_unimported_type
from mypy.typeanal import has_any_from_unimported_type, check_for_explicit_any
from mypy.types import (
Type, AnyType, CallableType, FunctionLike, Overloaded, TupleType, TypedDictType,
Instance, NoneTyp, strip_type, TypeType,
Expand Down Expand Up @@ -629,6 +629,8 @@ def is_implicit_any(t: Type) -> bool:
if has_any_from_unimported_type(arg_type):
prefix = "Argument {} to \"{}\"".format(idx + 1, fdef.name())
self.msg.unimported_type_becomes_any(prefix, arg_type, fdef)
check_for_explicit_any(fdef.type, self.options, self.is_typeshed_stub,
self.msg, context=fdef)
if name in nodes.reverse_op_method_set:
self.check_reverse_op_method(item, typ, name)
elif name in ('__getattr__', '__getattribute__'):
Expand Down Expand Up @@ -1215,6 +1217,7 @@ def visit_assignment_stmt(self, s: AssignmentStmt) -> None:
self.msg.unimported_type_becomes_any("A type on this line", AnyType(), s)
else:
self.msg.unimported_type_becomes_any("Type of variable", s.type, s)
check_for_explicit_any(s.type, self.options, self.is_typeshed_stub, self.msg, context=s)

if len(s.lvalues) > 1:
# Chained assignment (e.g. x = y = ...).
Expand Down
6 changes: 5 additions & 1 deletion mypy/checkexpr.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from typing import cast, Dict, Set, List, Tuple, Callable, Union, Optional

from mypy.errors import report_internal_error
from mypy.typeanal import has_any_from_unimported_type
from mypy.typeanal import has_any_from_unimported_type, check_for_explicit_any
from mypy.types import (
Type, AnyType, CallableType, Overloaded, NoneTyp, TypeVarDef,
TupleType, TypedDictType, Instance, TypeVarType, ErasedType, UnionType,
Expand Down Expand Up @@ -1692,6 +1692,8 @@ def visit_cast_expr(self, expr: CastExpr) -> Type:
self.msg.redundant_cast(target_type, expr)
if 'unimported' in options.disallow_any and has_any_from_unimported_type(target_type):
self.msg.unimported_type_becomes_any("Target type of cast", target_type, expr)
check_for_explicit_any(target_type, self.chk.options, self.chk.is_typeshed_stub, self.msg,
context=expr)
return target_type

def visit_reveal_type_expr(self, expr: RevealTypeExpr) -> Type:
Expand Down Expand Up @@ -2412,6 +2414,8 @@ def visit_namedtuple_expr(self, e: NamedTupleExpr) -> Type:
if ('unimported' in self.chk.options.disallow_any and
has_any_from_unimported_type(tuple_type)):
self.msg.unimported_type_becomes_any("NamedTuple type", tuple_type, e)
check_for_explicit_any(tuple_type, self.chk.options, self.chk.is_typeshed_stub,
self.msg, context=e)
# TODO: Perhaps return a type object type?
return AnyType()

Expand Down
2 changes: 1 addition & 1 deletion mypy/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ def type_check_only(sources: List[BuildSource], bin_dir: str, options: Options)
options=options)


disallow_any_options = ['unimported', 'expr', 'unannotated', 'decorated']
disallow_any_options = ['unimported', 'expr', 'unannotated', 'decorated', 'explicit']


def disallow_any_argument_type(raw_options: str) -> List[str]:
Expand Down
3 changes: 3 additions & 0 deletions mypy/messages.py
Original file line number Diff line number Diff line change
Expand Up @@ -879,6 +879,9 @@ def unimported_type_becomes_any(self, prefix: str, typ: Type, ctx: Context) -> N
self.fail("{} becomes {} due to an unfollowed import".format(prefix, self.format(typ)),
ctx)

def explicit_any(self, ctx: Context) -> None:
self.fail('Explicit "Any" is not allowed', ctx)

def unexpected_typeddict_keys(
self,
typ: TypedDictType,
Expand Down
38 changes: 35 additions & 3 deletions mypy/semanal.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
from contextlib import contextmanager

from typing import (
List, Dict, Set, Tuple, cast, TypeVar, Union, Optional, Callable, Iterator,
List, Dict, Set, Tuple, cast, TypeVar, Union, Optional, Callable, Iterator, Iterable
)

from mypy.nodes import (
Expand Down Expand Up @@ -80,12 +80,13 @@
NoneTyp, CallableType, Overloaded, Instance, Type, TypeVarType, AnyType,
FunctionLike, UnboundType, TypeList, TypeVarDef, TypeType,
TupleType, UnionType, StarType, EllipsisType, function_type, TypedDictType,
TypeQuery
TypeTranslator,
)
from mypy.nodes import implicit_module_attrs
from mypy.typeanal import (
TypeAnalyser, TypeAnalyserPass3, analyze_type_alias, no_subscript_builtin_alias,
TypeVariableQuery, TypeVarList, remove_dups, has_any_from_unimported_type
TypeVariableQuery, TypeVarList, remove_dups, has_any_from_unimported_type,
check_for_explicit_any
)
from mypy.exprtotype import expr_to_unanalyzed_type, TypeTranslationError
from mypy.sametypes import is_same_type
Expand Down Expand Up @@ -230,6 +231,7 @@ class SemanticAnalyzer(NodeVisitor):
loop_depth = 0 # Depth of breakable loops
cur_mod_id = '' # Current module id (or None) (phase 2)
is_stub_file = False # Are we analyzing a stub file?
is_typeshed_stub_file = False # Are we analyzing a typeshed stub file?
imports = None # type: Set[str] # Imported modules (during phase 2 analysis)
errors = None # type: Errors # Keeps track of generated errors
plugin = None # type: Plugin # Mypy plugin for special casing of library features
Expand Down Expand Up @@ -274,6 +276,7 @@ def visit_file(self, file_node: MypyFile, fnam: str, options: Options,
self.cur_mod_node = file_node
self.cur_mod_id = file_node.fullname()
self.is_stub_file = fnam.lower().endswith('.pyi')
self.is_typeshed_stub_file = self.errors.is_typeshed_file(file_node.path)
self.globals = file_node.names
self.patches = patches

Expand Down Expand Up @@ -339,6 +342,7 @@ def file_context(self, file_node: MypyFile, fnam: str, options: Options,
self.cur_mod_node = file_node
self.cur_mod_id = file_node.fullname()
self.is_stub_file = fnam.lower().endswith('.pyi')
self.is_typeshed_stub_file = self.errors.is_typeshed_file(file_node.path)
self.globals = file_node.names
if active_type:
self.enter_class(active_type.defn.info)
Expand Down Expand Up @@ -1017,6 +1021,8 @@ def analyze_base_classes(self, defn: ClassDef) -> None:
else:
prefix = "Base type"
self.msg.unimported_type_becomes_any(prefix, base, base_expr)
check_for_explicit_any(base, self.options, self.is_typeshed_stub_file, self.msg,
context=base_expr)

# Add 'object' as implicit base if there is no other base class.
if (not base_types and defn.fullname != 'builtins.object'):
Expand Down Expand Up @@ -1571,6 +1577,12 @@ def visit_assignment_stmt(self, s: AssignmentStmt) -> None:
allow_unnormalized=True)
if res and (not isinstance(res, Instance) or res.args):
# TODO: What if this gets reassigned?
check_for_explicit_any(res, self.options, self.is_typeshed_stub_file, self.msg,
context=s)
# when this type alias gets "inlined", the Any is not explicit anymore,
# so we need to replace it with non-explicit Anys
res = make_any_non_explicit(res)

name = s.lvalues[0]
node = self.lookup(name.name, name)
node.kind = TYPE_ALIAS
Expand Down Expand Up @@ -1835,6 +1847,9 @@ def process_newtype_declaration(self, s: AssignmentStmt) -> None:
self.fail(message.format(old_type), s)
return

check_for_explicit_any(old_type, self.options, self.is_typeshed_stub_file, self.msg,
context=s)

if 'unimported' in self.options.disallow_any and has_any_from_unimported_type(old_type):
self.msg.unimported_type_becomes_any("Argument 2 to NewType(...)", old_type, s)

Expand Down Expand Up @@ -1959,6 +1974,9 @@ def process_typevar_declaration(self, s: AssignmentStmt) -> None:
prefix = "Upper bound of type variable"
self.msg.unimported_type_becomes_any(prefix, upper_bound, s)

for t in values + [upper_bound]:
check_for_explicit_any(t, self.options, self.is_typeshed_stub_file, self.msg,
context=s)
# Yes, it's a valid type variable definition! Add it to the symbol table.
node = self.lookup(name, s)
node.kind = TVAR
Expand Down Expand Up @@ -2393,6 +2411,10 @@ def parse_typeddict_args(self, call: CallExpr,
'TypedDict() "total" argument must be True or False', call)
dictexpr = args[1]
items, types, ok = self.parse_typeddict_fields_with_types(dictexpr.items, call)
for t in types:
check_for_explicit_any(t, self.options, self.is_typeshed_stub_file, self.msg,
context=call)

if 'unimported' in self.options.disallow_any:
for t in types:
if has_any_from_unimported_type(t):
Expand Down Expand Up @@ -4390,3 +4412,13 @@ def find_fixed_callable_return(expr: Expression) -> Optional[CallableType]:
if isinstance(t.ret_type, CallableType):
return t.ret_type
return None


def make_any_non_explicit(t: Type) -> Type:
"""Replace all Any types within in with Any that has attribute 'explicit' set to False"""
return t.accept(MakeAnyNonExplicit())


class MakeAnyNonExplicit(TypeTranslator):
def visit_any(self, t: AnyType) -> Type:
return t.copy_modified(explicit=False)
35 changes: 34 additions & 1 deletion mypy/typeanal.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

from contextlib import contextmanager

from mypy.messages import MessageBuilder
from mypy.options import Options
from mypy.types import (
Type, UnboundType, TypeVarType, TupleType, TypedDictType, UnionType, Instance,
AnyType, CallableType, NoneTyp, DeletedType, TypeList, TypeVarDef, TypeVisitor,
Expand Down Expand Up @@ -166,7 +168,7 @@ def visit_unbound_type(self, t: UnboundType) -> Type:
elif fullname == 'builtins.None':
return NoneTyp()
elif fullname == 'typing.Any' or fullname == 'builtins.Any':
return AnyType()
return AnyType(explicit=True)
elif fullname == 'typing.Tuple':
if len(t.args) == 0 and not t.empty_tuple_index:
# Bare 'Tuple' is same as 'tuple'
Expand Down Expand Up @@ -754,6 +756,37 @@ def visit_callable_type(self, t: CallableType) -> TypeVarList:
return []


def check_for_explicit_any(typ: Optional[Type],
options: Options,
is_typeshed_stub: bool,
msg: MessageBuilder,
context: Context) -> None:
if ('explicit' in options.disallow_any and
not is_typeshed_stub and
typ and
has_explicit_any(typ)):
msg.explicit_any(context)


def has_explicit_any(t: Type) -> bool:
"""
Whether this type is or type it contains is an Any coming from explicit type annotation
"""
return t.accept(HasExplicitAny())


class HasExplicitAny(TypeQuery[bool]):
def __init__(self) -> None:
super().__init__(any)

def visit_any(self, t: AnyType) -> bool:
return t.explicit

def visit_typeddict_type(self, t: TypedDictType) -> bool:
# typeddict is checked during TypedDict declaration, so don't typecheck it here.
return False


def has_any_from_unimported_type(t: Type) -> bool:
"""Return true if this type is Any because an import was not followed.

Expand Down
26 changes: 23 additions & 3 deletions mypy/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -249,23 +249,46 @@ def serialize(self) -> JsonDict:
assert False, "Sythetic types don't serialize"


_dummy = object() # type: Any


class AnyType(Type):
"""The type 'Any'."""

def __init__(self,
implicit: bool = False,
from_unimported_type: bool = False,
explicit: bool = False,
line: int = -1,
column: int = -1) -> None:
super().__init__(line, column)
# Was this Any type was inferred without a type annotation?
# Note that this is not always the opposite of explicit.
# For instance, if "Any" comes from an unimported type,
# both explicit and implicit will be False
self.implicit = implicit
# Does this come from an unfollowed import? See --disallow-any=unimported option
self.from_unimported_type = from_unimported_type
# Does this Any come from an explicit type annotation?
self.explicit = explicit
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not always the opposite of implicit? If not, the specifics need to be made very clear in the comment.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, explicit is not always the opposite of implicit, which is confusing.

implicit refers to situations when Any was inferred because a variable or parameter was not annotated.
explicit refers to situations when there was an explicit Any annotation.

However, if this Any comes, for instance, from an unimported type, both explicit and implicit will be False.

It seems to me that implicit has a wrong name. It is currently used for --disallow-untyped-defs to see if this Any comes from a parameter or variable with no explicit type annotation.

I think instead of tweaking comments we should rename implicit. How do you feel about calling it from_unannotated?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added comment on implicit to make it more clear.


def accept(self, visitor: 'TypeVisitor[T]') -> T:
return visitor.visit_any(self)

def copy_modified(self,
implicit: bool = _dummy,
from_unimported_type: bool = _dummy,
explicit: bool = _dummy,
) -> 'AnyType':
if implicit is _dummy:
implicit = self.implicit
if from_unimported_type is _dummy:
from_unimported_type = self.from_unimported_type
if explicit is _dummy:
explicit = self.explicit
return AnyType(implicit=implicit, from_unimported_type=from_unimported_type,
explicit=explicit, line=self.line, column=self.column)

def serialize(self) -> JsonDict:
return {'.class': 'AnyType'}

Expand Down Expand Up @@ -509,9 +532,6 @@ def get_name(self) -> Optional[str]: pass
fallback = None # type: Instance


_dummy = object() # type: Any


FormalArgument = NamedTuple('FormalArgument', [
('name', Optional[str]),
('pos', Optional[int]),
Expand Down
Loading