Skip to content

UnicodeDecodeError on Windows with pyproject.toml #11058

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
domdfcoding opened this issue Sep 4, 2021 · 1 comment · Fixed by #12105
Closed

UnicodeDecodeError on Windows with pyproject.toml #11058

domdfcoding opened this issue Sep 4, 2021 · 1 comment · Fixed by #12105
Labels

Comments

@domdfcoding
Copy link
Contributor

Crash Report

On Windows (where the default encoding for open is usually not UTF-8), with a pyproject.toml file containing the mypy configuration, mypy crashes due to being unable to decode unicode characters.

The TOML specification requires files to be encoded in UTF-8, so it is valid to assume tools will correctly parse files containing unicode characters.

Traceback

Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\a\domdf_python_tools\domdf_python_tools\.tox\mypy\Scripts\mypy.EXE\__main__.py", line 7, in <module>
  File "D:\a\domdf_python_tools\domdf_python_tools\.tox\mypy\lib\site-packages\mypy\__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "mypy\main.py", line 87, in main
  File "mypy\main.py", line 165, in run_build
  File "mypy\build.py", line 179, in build
  File "mypy\build.py", line 229, in _build
  File "mypy\build.py", line 475, in load_plugins
  File "mypy\build.py", line 391, in load_plugins_from_config
  File "mypy\build.py", line 511, in find_config_file_line_number
  File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 220: character maps to <undefined>

This is caused by the following function, particularly the call to open on line 511 without an encoding being specified.

mypy/mypy/build.py

Lines 503 to 523 in 5c4aea3

def find_config_file_line_number(path: str, section: str, setting_name: str) -> int:
"""Return the approximate location of setting_name within mypy config file.
Return -1 if can't determine the line unambiguously.
"""
in_desired_section = False
try:
results = []
with open(path) as f:
for i, line in enumerate(f):
line = line.strip()
if line.startswith('[') and line.endswith(']'):
current_section = line[1:-1].strip()
in_desired_section = (current_section == section)
elif in_desired_section and re.match(r'{}\s*='.format(setting_name), line):
results.append(i + 1)
if len(results) == 1:
return results[0]
except OSError:
pass
return -1

To Reproduce

  • Use the following pyproject.toml file content:
[project]
description = "Factory ⸻ A code generator 🏭"

[tool.mypy]
  • Run mypy 0.910 on Windows

Your Environment

  • Mypy version used: 0.910
  • Mypy command-line flags: None
  • Mypy configuration options from mypy.ini (and other config files): None (empty tool.mypy section in pyproject.toml)
  • Python version used: 3.6 and 3.9
  • Operating system and version: Windows 10
@domdfcoding
Copy link
Contributor Author

This issue is still present in mypy 0.930

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant