Skip to content

Commit aa6e3ef

Browse files
authored
README.md: better document test_stubtest pitfall (#3793)
Document that the test will likely fail locally, to help cases like #3782
1 parent d539772 commit aa6e3ef

File tree

2 files changed

+11
-6
lines changed

2 files changed

+11
-6
lines changed

README.md

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -162,18 +162,23 @@ This test requires Python 3.5 or higher.
162162
Run using `(.venv3)$ python3 tests/stubtest_test.py`
163163

164164
This test compares the stdlib stubs against the objects at runtime. Because of
165-
this, the output depends on which version of Python it is run with.
165+
this, the output depends on which version of Python and on what kind of system
166+
it is run.
167+
Thus the easiest way to run this test is by enabling Travis CI on your fork;
168+
if you run it locally, it'll likely complain about system-specific
169+
differences (in e.g, `socket`) that the type system cannot capture.
166170
If you need a specific version of Python to repro a CI failure,
167-
[pyenv](https://github.com/pyenv/pyenv) can help (as can enabling Travis CI on
168-
your fork).
171+
[pyenv](https://github.com/pyenv/pyenv) can help.
169172

170173
Due to its dynamic nature, you may run into false positives. In this case, you
171174
can add to the whitelists for each affected Python version in
172175
`tests/stubtest_whitelists`. Please file issues for stubtest false positives
173176
at [mypy](https://github.com/python/mypy/issues).
174177

175178
To run stubtest against third party stubs, it's easiest to use stubtest
176-
directly. stubtest can also help you find things missing from the stubs.
179+
directly, with `(.venv3)$ python3 -m mypy.stubtest --custom-typeshed-dir
180+
<path-to-typeshed> <third-party-module>`.
181+
stubtest can also help you find things missing from the stubs.
177182

178183

179184
### flake8

tests/stubtest_test.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,8 @@ def run_stubtest(typeshed_dir: Path) -> int:
4444
subprocess.run(cmd, check=True)
4545
except subprocess.CalledProcessError as e:
4646
print(
47-
"\nNB: stubtest output depends on the Python version it is run with. See README.md "
48-
"for more details.\n"
47+
"\nNB: stubtest output depends on the Python version (and system) it is run with. "
48+
"See README.md for more details.\n"
4949
"NB: We only check positional-only arg accuracy for Python 3.8.\n"
5050
"If stubtest is complaining about 'unused whitelist entry' after your fix, please "
5151
"remove the entry from the whitelist file. Note you may have to do this for other "

0 commit comments

Comments
 (0)