Merge branch 'master' into refactor-test-discovery

Author: elazarg

.travis.yml

@@ -9,9 +9,6 @@ python:
   - "3.5.1"
   - "3.6"
   - "3.7-dev"
-  # Pypy build is disabled because it doubles the travis build time, and it rarely fails
-  # unless one one of the other builds fails.
-  # - "pypy3"
 
 install:
   - pip install -U pip setuptools wheel
@@ -20,5 +17,6 @@ install:
   - pip install .
 
 script:
-  - python runtests.py -j12 -x lint
+  - python runtests.py -j12 -x lint -x package
   - if [[ $TRAVIS_PYTHON_VERSION == '3.6' ]]; then flake8; fi
+  - if [[ $TRAVIS_PYTHON_VERSION == '3.5.1' ]]; then python runtests.py package; fi

README.md

@@ -25,23 +25,15 @@ What is mypy?
 -------------
 
 Mypy is an optional static type checker for Python.  You can add type
-hints to your Python programs using the standard for type
-annotations introduced in Python 3.5 ([PEP 484](https://www.python.org/dev/peps/pep-0484/)), and use mypy to
-type check them statically. Find bugs in your programs without even
-running them!
-
-The type annotation standard has also been backported to earlier
-Python 3.x versions.  Mypy supports Python 3.3 and later.
-XML based reports do not work on Python 3.3 and 3.4 on Windows.
-
-For Python 2.7, you can add annotations as comments (this is also
-specified in [PEP 484](https://www.python.org/dev/peps/pep-0484/)).
+hints ([PEP 484](https://www.python.org/dev/peps/pep-0484/)) to your
+Python programs, and use mypy to type check them statically.
+Find bugs in your programs without even running them!
 
 You can mix dynamic and static typing in your programs. You can always
 fall back to dynamic typing when static typing is not convenient, such
 as for legacy code.
 
-Here is a small example to whet your appetite:
+Here is a small example to whet your appetite (Python 3):
 
 ```python
 from typing import Iterator
@@ -52,11 +44,20 @@ def fib(n: int) -> Iterator[int]:
         yield a
         a, b = b, a + b
 ```
+See [the documentation](http://mypy.readthedocs.io/en/stable/introduction.html) for more examples.
+
+For Python 2.7, the standard annotations are written as comments:
+```python
+def is_palindrome(s):
+    # type: (str) -> bool
+    return s == s[::-1]
+```
+
+See [the documentation for Python 2 support](http://mypy.readthedocs.io/en/latest/python2.html).
 
 Mypy is in development; some features are missing and there are bugs.
 See 'Development status' below.
 
-
 Requirements
 ------------
 
@@ -105,7 +106,7 @@ Mypy can be integrated into popular IDEs:
 
 * Vim: [vim-mypy](https://github.com/Integralist/vim-mypy)
 * Emacs: using [Flycheck](https://github.com/flycheck/) and [Flycheck-mypy](https://github.com/lbolla/emacs-flycheck-mypy/issues)
-* Sublime Text: [SublimeLinter-contrib-mypy]
+* Sublime Text: [SublimeLinter-contrib-mypy](https://github.com/fredcallaway/SublimeLinter-contrib-mypy)
 * Atom: [linter-mypy](https://atom.io/packages/linter-mypy)
 * PyCharm: PyCharm integrates [its own implementation of PEP 484](https://www.jetbrains.com/help/pycharm/2017.1/type-hinting-in-pycharm.html).
 
@@ -157,9 +158,6 @@ In Windows, the script is generally installed in
 
     C:\>\Python34\python \Python34\Scripts\mypy PROGRAM
 
-If you are on Windows using Python 3.3 or 3.4, and would like to use XML reports
-download LXML from [PyPi](https://pypi.python.org/pypi/lxml).
-
 ### Working with `virtualenv`
 
 If you are using [`virtualenv`](https://virtualenv.pypa.io/en/stable/),

appveyor.yml

@@ -12,15 +12,17 @@ install:
     - "git config core.symlinks true"
     - "git reset --hard"
     - "%PYTHON%\\python.exe -m pip install -r test-requirements.txt"
+    - "C:\\Python27\\python.exe -m pip install typing"
     - "git submodule update --init typeshed"
     - "cd typeshed && git config core.symlinks true && git reset --hard && cd .."
     - "%PYTHON%\\python.exe -m pip install ."
 
 build: off
 
 test_script:
-    # Ignore lint (it's run in Travis)
-    - "%PYTHON%\\python.exe runtests.py -x lint"
+    # Ignore lint and mypy self check (both run in Travis)
+    - "%PYTHON%\\python.exe runtests.py -x lint -x package -x pytest"
+    - "%PYTHON%\\python.exe runtests.py pytest -p -k -p \"not (PythonEvaluationSuite and python2)\""
 
 skip_commits:
   files:

docs/source/cheat_sheet_py3.rst

@@ -239,7 +239,7 @@ Other stuff
        else:
            return sys.stdout
 
-   # forward references are useful if you want to referemce a class before it is designed
+   # forward references are useful if you want to reference a class before it is designed
    
    def f(foo: A) -> int:  # this will fail
        ...
@@ -304,4 +304,4 @@ Mypy brings limited support for PEP 526 annotations.
         def __init__(self) -> None:
             self.items: List[str] = []
    
-Please see :ref:`python-36` for more on mypy's compatability with Python 3.6's new features.
+Please see :ref:`python-36` for more on mypy's compatibility with Python 3.6's new features.

docs/source/class_basics.rst

@@ -158,30 +158,32 @@ Protocols and structural subtyping
 
 .. note::
 
-   The support for structural subtyping is still experimental. Some features
-   might be not yet implemented, mypy could pass unsafe code or reject
-   working code.
-
-There are two main type systems with respect to subtyping: nominal subtyping
-and structural subtyping. The *nominal* subtyping is based on class hierarchy,
-so that if class ``D`` inherits from class ``C``, then it is a subtype
-of ``C``. This type system is primarily used in mypy since it allows
-to produce clear and concise error messages, and since Python provides native
-``isinstance()`` checks based on class hierarchy. The *structural* subtyping
-however has its own advantages. In this system class ``D`` is a subtype
-of class ``C`` if the former has all attributes of the latter with
-compatible types.
-
-This type system is a static equivalent of duck typing, well known by Python
-programmers. Mypy provides an opt-in support for structural subtyping via
-protocol classes described in this section.
-See `PEP 544 <https://www.python.org/dev/peps/pep-0544/>`_ for
-specification of protocols and structural subtyping in Python.
-
-User defined protocols
-**********************
-
-To define a protocol class, one must inherit the special
+   Structural subtyping is experimental. Some things may not
+   work as expected. Mypy may pass unsafe code or it can reject
+   valid code.
+
+Mypy supports two ways of deciding whether two classes are compatible
+as types: nominal subtyping and structural subtyping. *Nominal*
+subtyping is strictly based on the class hierarchy. If class ``D``
+inherits class ``C``, it's also a subtype of ``C``. This form of
+subtyping is used by default in mypy, since it's easy to understand
+and produces clear and concise error messages, and since it matches
+how the native ``isinstance()`` check works -- based on class
+hierarchy. *Structural* subtyping can also be useful. Class ``D`` is
+a structural subtype of class ``C`` if the former has all attributes
+and methods of the latter, and with compatible types.
+
+Structural subtyping can be seen as a static equivalent of duck
+typing, which is well known to Python programmers. Mypy provides an
+opt-in support for structural subtyping via protocol classes described
+below.  See `PEP 544 <https://www.python.org/dev/peps/pep-0544/>`_ for
+the detailed specification of protocols and structural subtyping in
+Python.
+
+Simple user-defined protocols
+*****************************
+
+You can define a protocol class by inheriting the special
 ``typing_extensions.Protocol`` class:
 
 .. code-block:: python
@@ -191,37 +193,42 @@ To define a protocol class, one must inherit the special
 
    class SupportsClose(Protocol):
        def close(self) -> None:
-          ...
+          ...  # Explicit '...'
+
+   class Resource:  # No SupportsClose base class!
+       # ... some methods ...
 
-   class Resource:  # Note, this class does not have 'SupportsClose' base.
-       # some methods
        def close(self) -> None:
           self.resource.release()
 
-   def close_all(things: Iterable[SupportsClose]) -> None:
-       for thing in things:
-           thing.close()
+   def close_all(items: Iterable[SupportsClose]) -> None:
+       for item in items:
+           item.close()
 
-   close_all([Resource(), open('some/file')])  # This passes type check
+   close_all([Resource(), open('some/file')])  # Okay!
+
+``Resource`` is a subtype of the ``SupportClose`` protocol since it defines
+a compatible ``close`` method. Regular file objects returned by ``open()`` are
+similarly compatible with the protocol, as they support ``close()``.
 
 .. note::
 
-   The ``Protocol`` base class is currently provided in ``typing_extensions``
-   package. When structural subtyping is mature and
-   `PEP 544 <https://www.python.org/dev/peps/pep-0544/>`_ is accepted,
-   ``Protocol`` will be included in the ``typing`` module. As well, several
-   types such as ``typing.Sized``, ``typing.Iterable`` etc. will be made
-   protocols.
+   The ``Protocol`` base class is currently provided in the ``typing_extensions``
+   package. Once structural subtyping is mature and
+   `PEP 544 <https://www.python.org/dev/peps/pep-0544/>`_ has been accepted,
+   ``Protocol`` will be included in the ``typing`` module. Several library
+   types such as ``typing.Sized`` and ``typing.Iterable`` will also be changed
+   into protocols. They are currently treated as regular ABCs by mypy.
 
 Defining subprotocols
 *********************
 
-Subprotocols are also supported. Existing protocols can be extended
-and merged using multiple inheritance. For example:
+You can also define subprotocols. Existing protocols can be extended
+and merged using multiple inheritance. Example:
 
 .. code-block:: python
 
-   # continuing from previous example
+   # ... continuing from the previous example
 
    class SupportsRead(Protocol):
        def read(self, amount: int) -> bytes: ...
@@ -232,46 +239,47 @@ and merged using multiple inheritance. For example:
    class AdvancedResource(Resource):
        def __init__(self, label: str) -> None:
            self.label = label
+
        def read(self, amount: int) -> bytes:
            # some implementation
            ...
 
-   resource = None  # type: TaggedReadableResource
-
-   # some code
-
+   resource: TaggedReadableResource
    resource = AdvancedResource('handle with care')  # OK
 
-Note that inheriting from existing protocols does not automatically turn
-a subclass into a protocol, it just creates a usual (non-protocol) ABC that
-implements given protocols. The ``typing_extensions.Protocol`` base must always
-be explicitly present:
+Note that inheriting from an existing protocol does not automatically
+turn the subclass into a protocol -- it just creates a regular
+(non-protocol) ABC that implements the given protocol (or
+protocols). The ``typing_extensions.Protocol`` base class must always
+be explicitly present if you are defining a protocol:
 
 .. code-block:: python
 
    class NewProtocol(SupportsClose):  # This is NOT a protocol
        new_attr: int
 
    class Concrete:
-      new_attr = None  # type: int
+      new_attr: int = 0
+
       def close(self) -> None:
           ...
-   # Below is an error, since nominal subtyping is used by default
-   x = Concrete()  # type: NewProtocol  # Error!
+
+   # Error: nominal subtyping used by default
+   x: NewProtocol = Concrete()  # Error!
 
 .. note::
 
-   The `PEP 526 <https://www.python.org/dev/peps/pep-0526/>`_ variable
-   annotations can be used to declare protocol attributes. However, protocols
-   are also supported on Python 2.7 and Python 3.3+ with the help of type
-   comments and properties, see
-   `backwards compatibility in PEP 544 <https://www.python.org/dev/peps/pep-0544/#backwards-compatibility>`_.
+   You can use Python 3.6 variable annotations (`PEP 526
+   <https://www.python.org/dev/peps/pep-0526/>`_)
+   to declare protocol attributes.  On Python 2.7 and earlier Python 3
+   versions you can use type comments and properties.
 
 Recursive protocols
 *******************
 
-Protocols can be recursive and mutually recursive. This could be useful for
-declaring abstract recursive collections such as trees and linked lists:
+Protocols can be recursive (self-referential) and mutually
+recursive. This is useful for declaring abstract recursive collections
+such as trees and linked lists:
 
 .. code-block:: python
 
@@ -280,8 +288,10 @@ declaring abstract recursive collections such as trees and linked lists:
 
    class TreeLike(Protocol):
        value: int
+
        @property
        def left(self) -> Optional['TreeLike']: ...
+
        @property
        def right(self) -> Optional['TreeLike']: ...
 
@@ -296,9 +306,9 @@ declaring abstract recursive collections such as trees and linked lists:
 Using ``isinstance()`` with protocols
 *************************************
 
-To use a protocol class with ``isinstance()``, one needs to decorate it with
-a special ``typing_extensions.runtime`` decorator. It will add support for
-basic runtime structural checks:
+You can use a protocol class with ``isinstance()`` if you decorate it
+with the ``typing_extensions.runtime`` class decorator. The decorator
+adds support for basic runtime structural checks:
 
 .. code-block:: python
 
@@ -314,11 +324,9 @@ basic runtime structural checks:
 
    mug = Mug()
    if isinstance(mug, Portable):
-      use(mug.handles)  # Works statically and at runtime.
+      use(mug.handles)  # Works statically and at runtime
 
 .. note::
-   ``isinstance()`` is with protocols not completely safe at runtime.
+   ``isinstance()`` with protocols is not completely safe at runtime.
    For example, signatures of methods are not checked. The runtime
-   implementation only checks the presence of all protocol members
-   in object's MRO.
-
+   implementation only checks that all protocol members are defined.