Skip to content

dmypy crashes on checking a couple files #18019

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Redoubts opened this issue Oct 23, 2024 · 2 comments · Fixed by #18098
Closed

dmypy crashes on checking a couple files #18019

Redoubts opened this issue Oct 23, 2024 · 2 comments · Fixed by #18098
Labels
crash topic-daemon dmypy

Comments

@Redoubts
Copy link

Crash Report

While trying to make a minimal repro of another bug, I noticed this reliable crasher

Traceback

Traceback (most recent call last):
  File "mypy/dmypy_server.py", line 236, in serve
  File "mypy/dmypy_server.py", line 285, in run_command
  File "mypy/dmypy_server.py", line 353, in cmd_run
  File "mypy/dmypy_server.py", line 432, in check
  File "mypy/dmypy_server.py", line 698, in fine_grained_increment_follow_imports
  File "mypy/server/update.py", line 267, in update
  File "mypy/server/update.py", line 369, in update_one
  File "mypy/server/update.py", line 452, in update_module
  File "mypy/server/update.py", line 881, in propagate_changes_using_dependencies
  File "mypy/server/update.py", line 1025, in reprocess_nodes
  File "mypy/checker.py", line 535, in check_second_pass
  File "mypy/checker.py", line 540, in check_partial
  File "mypy/checker.py", line 555, in check_top_level
  File "mypy/nodes.py", line 1351, in accept
  File "mypy/checker.py", line 2937, in visit_assignment_stmt
  File "mypy/checker.py", line 2978, in check_type_alias_rvalue
  File "mypy/checkexpr.py", line 5850, in accept
  File "mypy/errors.py", line 1269, in report_internal_error
  File "mypy/checkexpr.py", line 5848, in accept
  File "mypy/nodes.py", line 2112, in accept
  File "mypy/checkexpr.py", line 3394, in visit_op_expr
  File "mypy/checkexpr.py", line 5850, in accept
  File "mypy/errors.py", line 1269, in report_internal_error
  File "mypy/checkexpr.py", line 5848, in accept
  File "mypy/nodes.py", line 2714, in accept
  File "mypy/checkexpr.py", line 4744, in visit_type_alias_expr
  File "mypy/checkexpr.py", line 4808, in alias_type_in_runtime_context
  File "mypy/checker.py", line 6955, in named_generic_type
  File "mypy/checker.py", line 6962, in lookup_typeinfo
  File "mypy/checker.py", line 7040, in lookup_qualified
KeyError: 'types'

To Reproduce

% touch x.py y.py                                                                            

% dmypy --status-file ./dmypy.json run --timeout=86400 -- --cache-dir=/dev/null x.py        
Daemon started
Success: no issues found in 1 source file

% dmypy --status-file ./dmypy.json run --timeout=86400 -- --cache-dir=/dev/null y.py
Success: no issues found in 1 source file

% dmypy --status-file ./dmypy.json run --timeout=86400 -- --cache-dir=/dev/null x.py
<venv>/lib/python3.11/site-packages/mypy/typeshed/stdlib/_codecs.pyi:12: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.13.0
<venv>/lib/python3.11/site-packages/mypy/typeshed/stdlib/_codecs.pyi:12: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.13.0
Daemon crashed!

Your Environment

  • Mypy version used: 1.13
  • Mypy command-line flags: --status-file ./dmypy.json run --timeout=86400 -- --cache-dir=/dev/null
  • Mypy configuration options from mypy.ini (and other config files):
  • Python version used: 3.11
  • Operating system and version: Debian 11
@Redoubts Redoubts added the crash label Oct 23, 2024
@JukkaL JukkaL added the topic-daemon dmypy label Oct 23, 2024
@svalentin
Copy link
Collaborator

Can repro. The full exception when running dmypy as module with a fresh master checkout:

svalentin-coder-main:~/src/mypy (master)$ python -m mypy.dmypy --version
dmypy 1.14.0+dev.9e689593c9b99555631c2e2bcc05cb5481a557fb
svalentin-coder-main:~/src/mypy (master)$ touch x.py y.py
svalentin-coder-main:~/src/mypy (master)$ python -m mypy.dmypy --status-file ./dmypy.json run --timeout=86400 -- --cache-dir=/dev/null x.py
Daemon started
Success: no issues found in 1 source file
svalentin-coder-main:~/src/mypy (master)$ python -m mypy.dmypy --status-file ./dmypy.json run --timeout=86400 -- --cache-dir=/dev/null y.py
Success: no issues found in 1 source file
svalentin-coder-main:~/src/mypy (master)$ python -m mypy.dmypy --status-file ./dmypy.json run --timeout=86400 -- --cache-dir=/dev/null x.py
/home/svalentin/src/mypy/mypy/typeshed/stdlib/_codecs.pyi:12: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.14.0+dev.9e689593c9b99555631c2e2bcc05cb5481a557fb
/home/svalentin/src/mypy/mypy/typeshed/stdlib/_codecs.pyi:12: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.14.0+dev.9e689593c9b99555631c2e2bcc05cb5481a557fb
Daemon crashed!
Traceback (most recent call last):
  File "/home/svalentin/src/mypy/mypy/dmypy_server.py", line 236, in serve
    resp = self.run_command(command, data)
  File "/home/svalentin/src/mypy/mypy/dmypy_server.py", line 285, in run_command
    ret = method(self, **data)
  File "/home/svalentin/src/mypy/mypy/dmypy_server.py", line 353, in cmd_run
    return self.check(sources, export_types, is_tty, terminal_width)
  File "/home/svalentin/src/mypy/mypy/dmypy_server.py", line 432, in check
    messages = self.fine_grained_increment_follow_imports(
  File "/home/svalentin/src/mypy/mypy/dmypy_server.py", line 698, in fine_grained_increment_follow_imports
    messages = fine_grained_manager.update([], to_delete)
  File "/home/svalentin/src/mypy/mypy/server/update.py", line 267, in update
    result = self.update_one(
  File "/home/svalentin/src/mypy/mypy/server/update.py", line 369, in update_one
    result = self.update_module(next_id, next_path, next_id in removed_set, followed)
  File "/home/svalentin/src/mypy/mypy/server/update.py", line 452, in update_module
    remaining += propagate_changes_using_dependencies(
  File "/home/svalentin/src/mypy/mypy/server/update.py", line 881, in propagate_changes_using_dependencies
    triggered |= reprocess_nodes(manager, graph, id, nodes, deps, processed_targets)
  File "/home/svalentin/src/mypy/mypy/server/update.py", line 1025, in reprocess_nodes
    more = checker.check_second_pass(nodes)
  File "/home/svalentin/src/mypy/mypy/checker.py", line 547, in check_second_pass
    self.check_partial(node)
  File "/home/svalentin/src/mypy/mypy/checker.py", line 552, in check_partial
    self.check_top_level(node)
  File "/home/svalentin/src/mypy/mypy/checker.py", line 567, in check_top_level
    d.accept(self)
  File "/home/svalentin/src/mypy/mypy/nodes.py", line 1364, in accept
    return visitor.visit_assignment_stmt(self)
  File "/home/svalentin/src/mypy/mypy/checker.py", line 2978, in visit_assignment_stmt
    self.check_type_alias_rvalue(s)
  File "/home/svalentin/src/mypy/mypy/checker.py", line 3019, in check_type_alias_rvalue
    alias_type = self.expr_checker.accept(s.rvalue)
  File "/home/svalentin/src/mypy/mypy/checkexpr.py", line 5875, in accept
    report_internal_error(
  File "/home/svalentin/src/mypy/mypy/errors.py", line 1288, in report_internal_error
    raise err
  File "/home/svalentin/src/mypy/mypy/checkexpr.py", line 5873, in accept
    typ = node.accept(self)
  File "/home/svalentin/src/mypy/mypy/nodes.py", line 2127, in accept
    return visitor.visit_op_expr(self)
  File "/home/svalentin/src/mypy/mypy/checkexpr.py", line 3403, in visit_op_expr
    return self.accept(e.analyzed)
  File "/home/svalentin/src/mypy/mypy/checkexpr.py", line 5875, in accept
    report_internal_error(
  File "/home/svalentin/src/mypy/mypy/errors.py", line 1288, in report_internal_error
    raise err
  File "/home/svalentin/src/mypy/mypy/checkexpr.py", line 5873, in accept
    typ = node.accept(self)
  File "/home/svalentin/src/mypy/mypy/nodes.py", line 2729, in accept
    return visitor.visit_type_alias_expr(self)
  File "/home/svalentin/src/mypy/mypy/checkexpr.py", line 4769, in visit_type_alias_expr
    return self.alias_type_in_runtime_context(alias.node, ctx=alias, alias_definition=True)
  File "/home/svalentin/src/mypy/mypy/checkexpr.py", line 4833, in alias_type_in_runtime_context
    return self.chk.named_generic_type("types.UnionType", item.items)
  File "/home/svalentin/src/mypy/mypy/checker.py", line 7037, in named_generic_type
    info = self.lookup_typeinfo(name)
  File "/home/svalentin/src/mypy/mypy/checker.py", line 7044, in lookup_typeinfo
    sym = self.lookup_qualified(fullname)
  File "/home/svalentin/src/mypy/mypy/checker.py", line 7122, in lookup_qualified
    n = self.modules[parts[0]]
KeyError: 'types'

@svalentin
Copy link
Collaborator

svalentin commented Oct 23, 2024 •
edited
Loading

Git bisected this error to 98a22c4
Before that commit, running dmypy on x.py, y.py and then x.py worked.

Though, it's most likely not that commit at fault by iteself, but an interraction with dmypy, or incremental checking.
The second check for x.py takes noticeably longer even before that commit. So it's clearly doing something it shouldn't.

@ilevkivskyi ilevkivskyi mentioned this issue Nov 3, 2024
Merged
JukkaL pushed a commit that referenced this issue Nov 4, 2024
@ilevkivskyi
Fix couple crashes in dmypy (#18098)
976d105 
Fixes #18019
Fixes #17775

These two are essentially variations of the same thing. Instead of
adding e.g. `types` to `SENSITIVE_INTERNAL_MODULES` (which would be
fragile and re-introduce same crashes whenever we add a new "core"
module) I add _all stdlib modules_. The only scenario when stdlib
changes is when a version of mypy changes, and in this case the daemon
will be (or should be) restarted anyway.

While adding tests for these I noticed a discrepancy in
`--follow-imports=normal` in the daemon: the files explicitly added on
the command line should be always treated as changed, since otherwise we
will not detect errors if a file was removed from command line in an
intermediate run.

Finally the tests also discovered a spurious error when cache is
disabled (via `/dev/null`).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
crash topic-daemon dmypy
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix couple crashes in dmypy ilevkivskyi/mypy
3 participants
@svalentin @JukkaL @Redoubts