From 3148825b42da99d0ba98b827bd8d2991a501755c Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Wed, 3 May 2017 15:03:51 -0700 Subject: [PATCH 1/6] Docs for flexible Callable --- docs/source/kinds_of_types.rst | 124 +++++++++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) diff --git a/docs/source/kinds_of_types.rst b/docs/source/kinds_of_types.rst index 67b4b42a5074..6b090827f106 100644 --- a/docs/source/kinds_of_types.rst +++ b/docs/source/kinds_of_types.rst @@ -188,6 +188,130 @@ using bidirectional type inference: If you want to give the argument or return value types explicitly, use an ordinary, perhaps nested function definition. +Extended Callable types +*********************** + +As an experimental mypy extension, you can specify ``Callable`` types +that support keyword arguments, optional arguments, and more. The +"argument" part of the ``Callable`` specification (the part specifying +the types of the arguments) can be a list of "argument specifiers" +instead of a list of types. This allows one to more closely emulate +the full range of possibilities given by the ``def`` statement in +Python. + +As an example, here's a complicated function definition and the +corresponding ``Callable``: + +.. code-block:: python + + from typing import Callable + from mypy_extensions import (Arg, DefaultArg, NamedArg, + DefaultNamedArg, VarArg, KwArg) + + def func(__a: int, # This convention is for nameless arguments + b: int, + c: int = 0, + *args: int, + d: int, + e: int = 0, + **kwargs: int) -> int: + ... + + F = Callable[[Arg(int), # No argument name given + Arg(int, 'b'), + DefaultArg(int, 'c'), + VarArg(int), + NamedArg(int, 'd'), + DefaultNamedArg(int, 'e'), + KwArg(int)], + int] + + f: F = func + +Argument specifiers are special function calls that can specify the +following aspects of an argument: + +- its type (the only thing that the basic format supports) + +- its name (if it has one) + +- whether it may be omitted + +- whether it may or must be passed using a keyword + +- whether it is a ``*args`` argument (representing the remaining + positional arguments) + +- whether it is a ``**kwargs`` argument (representing the remaining + keyword arguments) + +The following functions are available in ``mypy_extensions`` for this +purpose: + +.. code-block:: python + + def Arg(type=Any, name=None): + # A normal, mandatory, positional argument. + # If the name is specified it may be passed as a keyword. + + def DefaultArg(type=Any, name=None): + # An optional positional argument (i.e. with a default value). + # If the name is specified it may be passed as a keyword. + + def NamedArg(type=Any, name=None): + # A mandatory keyword-only argument. + + def DefaultNamedArg(type=Any, name=None): + # An optional keyword-only argument (i.e. with a default value). + + def VarArg(type=Any): + # A *args-style variadic positional argument. + # A single VarArg() specifier represents all remaining + # positional arguments. + + def KwArg(type=Any): + # A **kwargs-style variadic keyword argument. + # A single KwArg() specifier represents all remaining + # keyword arguments. + +In all cases, the ``type`` argument defaults to ``Any``, and if the +``name`` argument is omitted the argument has no name (the name is +required for ``NamedArg`` and ``DefaultNamedArg``). A basic +``Callable`` such as + +.. code-block:: python + + MyFunc = Callable[[int, str, int], float] + +is equivalent to the following: + +.. code-block:: python + + MyFunc = Callable[[Arg(int), Arg(str), Arg(int)], float] + +A ``Callable`` with unspecified argument types, such as + +.. code-block:: python + + MyOtherFunc = Callable[..., int] + +is equivalent to + +.. code-block:: python + + MyOtherFunc = Callable[[VarArg(), KwArg()], int] + +.. note:: + + This feature is experimental. Details of the implementation may + change and there may be unknown limitations. **IMPORTANT:** At + runtime, each of the functions above just returns its ``type`` + argument, so the information contained in the argument specifiers + is not available at runtime. This limitation is necessary for + backwards compatibility with the existing ``typing.py`` module as + present in the Python 3.5+ standard library and distributed via + PyPI. + .. _union-types: Union types From 19b533bb6a4a678e392457966ee9edbe602c982e Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Wed, 3 May 2017 15:12:08 -0700 Subject: [PATCH 2/6] Document --cache-dir=/dev/null --- docs/source/command_line.rst | 9 ++++++++- docs/source/config_file.rst | 4 +++- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/docs/source/command_line.rst b/docs/source/command_line.rst index dcd46dcd9aa7..fbab49a0f2c5 100644 --- a/docs/source/command_line.rst +++ b/docs/source/command_line.rst @@ -301,7 +301,14 @@ Here are some more useful flags: - ``--incremental`` is an experimental option that enables incremental type checking. When enabled, mypy caches results from previous runs to speed up type checking. Incremental mode can help when most parts - of your program haven't changed since the previous mypy run. + of your program haven't changed since the previous mypy run. A + companion flag is ``--cache-dir DIR``, which specifies where the + cache files are written. By default this is ``.mypy_cache`` in the + current directory. While the cache is only read in incremental + mode, it is written even in non-incremental mode, in order to "warm" + the cache. To disable writing the cache, use + ``--cache-dir=/dev/null`` (UNIX) or ``--cache-dir=nul`` (Windows). + Cache files belonging to a different mypy version are ignored. - ``--python-version X.Y`` will make mypy typecheck your code as if it were run under Python version X.Y. Without this option, mypy will default to using diff --git a/docs/source/config_file.rst b/docs/source/config_file.rst index 504d62b05347..84c750ff1f07 100644 --- a/docs/source/config_file.rst +++ b/docs/source/config_file.rst @@ -97,7 +97,9 @@ The following global flags may only be set in the global section module cache. - ``cache_dir`` (string, default ``.mypy_cache``) stores module cache - info in the given folder in incremental mode. + info in the given folder in incremental mode. The cache is only + read in incremental mode, but it is always written unless the value + is set to ``/dev/null`` (UNIX) or ``nul`` (Windows). - ``show_error_context`` (Boolean, default False) shows context notes before errors. From 52de25d2b1a7a8ac06552ad5168b7f1f89db5715 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Wed, 3 May 2017 15:30:44 -0700 Subject: [PATCH 3/6] Document quick mode --- docs/source/command_line.rst | 18 ++++++++++++++++-- docs/source/config_file.rst | 14 +++++++++----- 2 files changed, 25 insertions(+), 7 deletions(-) diff --git a/docs/source/command_line.rst b/docs/source/command_line.rst index fbab49a0f2c5..d88ff36ae3f9 100644 --- a/docs/source/command_line.rst +++ b/docs/source/command_line.rst @@ -298,8 +298,10 @@ Here are some more useful flags: the base class even though that may not actually be the case. This flag makes mypy raise an error instead. -- ``--incremental`` is an experimental option that enables incremental - type checking. When enabled, mypy caches results from previous runs +.. _incremental: + +- ``--incremental`` is an experimental option that enables a module + cache. When enabled, mypy caches results from previous runs to speed up type checking. Incremental mode can help when most parts of your program haven't changed since the previous mypy run. A companion flag is ``--cache-dir DIR``, which specifies where the @@ -310,6 +312,18 @@ Here are some more useful flags: ``--cache-dir=/dev/null`` (UNIX) or ``--cache-dir=nul`` (Windows). Cache files belonging to a different mypy version are ignored. +.. _quick: + +- ``--quick-and-dirty`` is an experimental, unsafe variant of + :ref:`incremental mode `. Quick mode is faster than + regular incremental mode, because it only re-checks modules that + were modified since their cache file was last written (regular + incremental mode also re-checks all modules that depend on one or + more modules that were re-checked). Quick mode is unsafe because it + may miss problems caused by a change in a dependency. Quick mode + updates the cache, but regular incremental mode ignores cache files + written by quick mode. + - ``--python-version X.Y`` will make mypy typecheck your code as if it were run under Python version X.Y. Without this option, mypy will default to using whatever version of Python is running mypy. Note that the ``-2`` and diff --git a/docs/source/config_file.rst b/docs/source/config_file.rst index 84c750ff1f07..0accd57411cd 100644 --- a/docs/source/config_file.rst +++ b/docs/source/config_file.rst @@ -93,13 +93,17 @@ The following global flags may only be set in the global section - ``dump_inference_stats`` (Boolean, default False) dumps stats about type inference. -- ``incremental`` (Boolean, default False) enables the experimental - module cache. +- ``incremental`` (Boolean, default False) enables :ref:`incremental + mode `. - ``cache_dir`` (string, default ``.mypy_cache``) stores module cache - info in the given folder in incremental mode. The cache is only - read in incremental mode, but it is always written unless the value - is set to ``/dev/null`` (UNIX) or ``nul`` (Windows). + info in the given folder in :ref:`incremental mode `. + The cache is only read in incremental mode, but it is always written + unless the value is set to ``/dev/null`` (UNIX) or ``nul`` + (Windows). + +- ``quick_and_dirty`` (Boolean, default False) enables :ref:`quick + mode `. - ``show_error_context`` (Boolean, default False) shows context notes before errors. From 4064a3ab3735e728c10f561965eea722ef72d7e7 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Wed, 3 May 2017 15:31:50 -0700 Subject: [PATCH 4/6] Update usage message to the most recent one --- docs/source/command_line.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/source/command_line.rst b/docs/source/command_line.rst index d88ff36ae3f9..3dfb124bf36c 100644 --- a/docs/source/command_line.rst +++ b/docs/source/command_line.rst @@ -14,14 +14,15 @@ flag (or its long form ``--help``):: [--disallow-untyped-calls] [--disallow-untyped-defs] [--check-untyped-defs] [--disallow-subclassing-any] [--warn-incomplete-stub] [--warn-redundant-casts] - [--warn-no-return] [--warn-unused-ignores] [--show-error-context] - [-i] [--cache-dir DIR] [--strict-optional] - [--strict-optional-whitelist [GLOB [GLOB ...]]] [--strict] + [--no-warn-no-return] [--warn-return-any] [--warn-unused-ignores] + [--show-error-context] [-i] [--quick-and-dirty] [--cache-dir DIR] + [--strict-optional] + [--strict-optional-whitelist [GLOB [GLOB ...]]] [--junit-xml JUNIT_XML] [--pdb] [--show-traceback] [--stats] [--inferstats] [--custom-typing MODULE] [--custom-typeshed-dir DIR] [--scripts-are-modules] [--config-file CONFIG_FILE] [--show-column-numbers] - [--find-occurrences CLASS.MEMBER] + [--find-occurrences CLASS.MEMBER] [--strict] [--strict-boolean] [--cobertura-xml-report DIR] [--html-report DIR] [--linecount-report DIR] [--linecoverage-report DIR] [--memory-xml-report DIR] [--old-html-report DIR] From 8101de766e36b6581d58c0cb8cd42178a7a2c3fe Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Wed, 3 May 2017 13:56:32 -0700 Subject: [PATCH 5/6] Add some docs and an example for ClassVar. This goes in the 3.6 section because it only works for Python 3.6. --- docs/source/python36.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/docs/source/python36.rst b/docs/source/python36.rst index 46a0bbf68713..9b4c86ab4fd7 100644 --- a/docs/source/python36.rst +++ b/docs/source/python36.rst @@ -16,6 +16,7 @@ now have type annotations using either of the two forms: .. code-block:: python + from typing import Optionsl foo: Optional[int] bar: List[str] = [] @@ -26,6 +27,27 @@ Mypy fully supports this syntax, interpreting them as equivalent to foo = None # type: Optional[int] bar = [] # type: List[str] +An additional feature defined in PEP 526 is also supported: you can +mark names intended to be used as class variables with ``ClassVar``. +In a pinch you can also use ClassVar in ``# type`` comments. +Example: + +.. code-block:: python + + from typing import ClassVar + + class C: + x: int # instance variable + y: ClassVar[int] # class variable + z = None # type: ClassVar[int] + + def foo(self) -> None: + self.x = 0 # OK + self.y = 0 # Error: Cannot assign to class variable "y" via instance + + C.y = 0 # This is OK + + Literal string formatting (`PEP 498 `_) --------------------------------------------------------------------------------- From c186459df0723587fd2a6e05b33e403a859d1283 Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Wed, 3 May 2017 16:22:39 -0700 Subject: [PATCH 6/6] Address code review --- docs/source/kinds_of_types.rst | 22 +++++++++++----------- docs/source/python36.rst | 2 +- 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/source/kinds_of_types.rst b/docs/source/kinds_of_types.rst index 6b090827f106..f7c784c4b406 100644 --- a/docs/source/kinds_of_types.rst +++ b/docs/source/kinds_of_types.rst @@ -192,12 +192,12 @@ Extended Callable types *********************** As an experimental mypy extension, you can specify ``Callable`` types -that support keyword arguments, optional arguments, and more. The -"argument" part of the ``Callable`` specification (the part specifying -the types of the arguments) can be a list of "argument specifiers" -instead of a list of types. This allows one to more closely emulate -the full range of possibilities given by the ``def`` statement in -Python. +that support keyword arguments, optional arguments, and more. Where +you specify the arguments of a Callable, you can choose to supply just +the type of a nameless positional argument, or an "argument specifier" +representing a more complicated form of argument. This allows one to +more closely emulate the full range of possibilities given by the +``def`` statement in Python. As an example, here's a complicated function definition and the corresponding ``Callable``: @@ -217,7 +217,7 @@ corresponding ``Callable``: **kwargs: int) -> int: ... - F = Callable[[Arg(int), # No argument name given + F = Callable[[int, # Or Arg(int) Arg(int, 'b'), DefaultArg(int, 'c'), VarArg(int), @@ -226,7 +226,7 @@ corresponding ``Callable``: KwArg(int)], int] - f: F = func + f: F = func Argument specifiers are special function calls that can specify the following aspects of an argument: @@ -295,7 +295,7 @@ A ``Callable`` with unspecified argument types, such as MyOtherFunc = Callable[..., int] -is equivalent to +is (roughly) equivalent to .. code-block:: python @@ -304,8 +304,8 @@ is equivalent to .. note:: This feature is experimental. Details of the implementation may - change and there may be unknown limitations. **IMPORTANT:** At - runtime, each of the functions above just returns its ``type`` + change and there may be unknown limitations. **IMPORTANT:** + Each of the functions above currently just returns its ``type`` argument, so the information contained in the argument specifiers is not available at runtime. This limitation is necessary for backwards compatibility with the existing ``typing.py`` module as diff --git a/docs/source/python36.rst b/docs/source/python36.rst index 9b4c86ab4fd7..12ec35ca6b3a 100644 --- a/docs/source/python36.rst +++ b/docs/source/python36.rst @@ -16,7 +16,7 @@ now have type annotations using either of the two forms: .. code-block:: python - from typing import Optionsl + from typing import Optional foo: Optional[int] bar: List[str] = []