mypy returns invalid type: "bytes" in classes that define a bytes instance attribute

[adewes]

When trying to add type annotations to a project (https://github.com/DCSO/flor), mypy would complain that bytes was not a valid type, which was quite puzzling to say the least. After some head scratching and searching in the issue log I was able to reproduce the problem with a simple module (Python 3.5.2+ with mypy 0.511):

class Foo(object):

    def __init__(self) -> None:
        self.bytes = bytearray(b"foo")

    def bar(self, f : bytes):
        return f + b"test"

foo = Foo()

foo.bar(b"test")

Running mypy example.py returns the following error messages:

example.py:6: error: Invalid type "bytes"
example.py:7: error: Unsupported left operand type for + (object)

When renaming self.bytes to self._bytes, no issues are reported. It seems that mypy mistakes self.bytes for a class variable, although it is an instance variable and not visible at the declaration time of function bar.

[emmatyping]

Thanks for reporting this! I reproduced the issue on 0.511, what's more concerning is that this is a crash on master!

test.py:6: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.520-dev-005fd6b357b995f13e3d90293cabf3c8c8c2a952
Traceback (most recent call last):
  File "C:\Python36\Scripts\mypy-script.py", line 11, in <module>
    load_entry_point('mypy', 'console_scripts', 'mypy')()
  File "c:\users\ethanhs\documents\mypy\mypy\__main__.py", line 7, in console_entry
    main(None)
  File "c:\users\ethanhs\documents\mypy\mypy\main.py", line 50, in main
    res = type_check_only(sources, bin_dir, options)
  File "c:\users\ethanhs\documents\mypy\mypy\main.py", line 97, in type_check_only
    options=options)
  File "c:\users\ethanhs\documents\mypy\mypy\build.py", line 196, in build
    graph = dispatch(sources, manager)
  File "c:\users\ethanhs\documents\mypy\mypy\build.py", line 1769, in dispatch
    process_graph(graph, manager)
  File "c:\users\ethanhs\documents\mypy\mypy\build.py", line 2012, in process_graph
    process_stale_scc(graph, scc, manager)
  File "c:\users\ethanhs\documents\mypy\mypy\build.py", line 2107, in process_stale_scc
    graph[id].semantic_analysis()
  File "c:\users\ethanhs\documents\mypy\mypy\build.py", line 1664, in semantic_analysis
    self.manager.semantic_analyzer.visit_file(self.tree, self.xpath, self.options, patches)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 295, in visit_file
    self.accept(d)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 3636, in accept
    node.accept(self)
  File "c:\users\ethanhs\documents\mypy\mypy\nodes.py", line 754, in accept
    return visitor.visit_class_def(self)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 668, in visit_class_def
    defn.defs.accept(self)
  File "c:\users\ethanhs\documents\mypy\mypy\nodes.py", line 815, in accept
    return visitor.visit_block(self)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 1522, in visit_block
    self.accept(s)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 3636, in accept
    node.accept(self)
  File "c:\users\ethanhs\documents\mypy\mypy\nodes.py", line 565, in accept
    return visitor.visit_func_def(self)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 418, in visit_func_def
    self.analyze_function(defn)
  File "c:\users\ethanhs\documents\mypy\mypy\semanal.py", line 598, in analyze_function
    defn.type = self.type_analyzer().visit_callable_type(defn.type, nested=False)
  File "c:\users\ethanhs\documents\mypy\mypy\typeanal.py", line 331, in visit_callable_type
    ret = t.copy_modified(arg_types=self.anal_array(t.arg_types, nested=nested),
  File "c:\users\ethanhs\documents\mypy\mypy\typeanal.py", line 520, in anal_array
    res.append(self.anal_type(t, nested))
  File "c:\users\ethanhs\documents\mypy\mypy\typeanal.py", line 527, in anal_type
    return t.accept(self)
  File "c:\users\ethanhs\documents\mypy\mypy\types.py", line 189, in accept
    return visitor.visit_unbound_type(self)
  File "c:\users\ethanhs\documents\mypy\mypy\typeanal.py", line 166, in visit_unbound_type
    tvar_def = self.tvar_scope.get_binding(sym)
  File "c:\users\ethanhs\documents\mypy\mypy\tvar_scope.py", line 71, in get_binding
    assert fullname is not None
AssertionError:
test.py:6: note: use --pdb to drop into pdb

As expected, renaming self.bytes resolved it.

[ilevkivskyi]

The crash should be not hard to fix.
@JukkaL If necessary, I can take a look at it right now, otherwise I would prefer to wait until after the release.

[JukkaL]

Let's wait until the release since this also happens on 0.511.

[JukkaL]

Ah so it's an error on 0.511 but a crash on master? Hmm I still want to move forward with the release, but if the fix is simple we may release 0.521 later this week.

[gvanrossum]

There are probably a few more candidates for 0.521...

…

On Mon, Jul 10, 2017 at 2:02 PM, Jukka Lehtosalo ***@***.***> wrote: Ah so it's an error on 0.511 but a crash on master? Hmm I still want to move forward with the release, but if the fix is simple we may release 0.521 later this week. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3687 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACwrMnSDeVIoFyLMlHa435jsra4ctQ9tks5sMpFxgaJpZM4OTT-K> .

-- --Guido van Rossum (python.org/~guido)

[JukkaL]

Here's a simpler example that generates the crash:

class C:
    def __init__(self) -> None:
        self.int = ''

    def f(self, f: int) -> None:
        f.x

Since the original issue hasn't been reported previously, this may be a fairly rare thing. The original error also was tricky to # type: ignore so likely there isn't a lot of code that has the error ignored (which likely would still cause a crash, however).

[ilevkivskyi]

OK, here is a (simple) fix #3700.

[adewes]

Thanks for handling this so quickly and efficiently! I gave a talk on type annotations at EuroPython 2017 and was happy to point out how great the support of the MyPy team is. If you're interested, here are the slides:

https://www.slideshare.net/japh44/type-annotations-in-python-whats-whys-and-wows

I also looked at the usage of type hints in the wild (analyzing the top 1000 Python repos on Github), which might be interesting to you:

https://github.com/adewes/type-annotations-in-the-wild

Thanks anyway, really looking forward to using mypy regularly in my work.

[gvanrossum]

Thanks! I am impressed with your talk. Thanks for spreading the word for us!