Add test README #3014
Add test README #3014
Conversation
test-data/unit/README.md
Outdated
|
|
||
| To add a simple unit test for a new feature you developed, open or create a | ||
| `test-data/unit/check-*.test` file with a name that roughly relates to the | ||
| feature you added (or create a new file). |
There was a problem hiding this comment.
you already said "or create" on line 8 above
test-data/unit/README.md
Outdated
| # indicated file (see Fixtures section for details) | ||
| [builtins fixtures/isinstancelist.pyi] | ||
|
|
||
| # the optional [out] indicates that any text after it contains the expected |
There was a problem hiding this comment.
Should we say that "E: " is preferred because it makes it easier to associate the errors with the code generating them at a glance, and it makes it easier to change the code of the test without having to change line numbers in [out]?
Also would be helpful to add an example out line.
test-data/unit/README.md
Outdated
| - The builtins used by default in unit tests live in | ||
| `test-data/unit/lib-stub`. | ||
|
|
||
| - Individual tests cases can override the stubs by using `[builtins |
pkch
force-pushed
the
test-readme
branch
2 times, most recently
from
d7e542a to
2508e0d
Compare
|
Also it will be very helpful to add some info on how to run tests (for example how to run a single test, or a selection of tests, etc). |
test-data/unit/README.md
Outdated
| any text after it contains the expected type checking error messages. | ||
| usually, "E: " is preferred because it makes it easier to associate the | ||
| errors with the code generating them at a glance, and to change the code of | ||
| the test without having to change line numbers in [out] |
There was a problem hiding this comment.
I would put [out] in backquotes here, and also [opt] and [builtins] above.
@pkch Ah sorry I didn't notice the last part about running tests with "(this should be expanded)". |
|
@ilevkivskyi hmm should I move this section from the main README.md to this one? |
|
@pkch I would prefer to have that section in test README. In the main one it will be overlooked by most people. Plus this will simplify the main README, which is probably too large now. |
test-data/unit/README.md
Outdated
| zzz: str # E: Name 'zzz' already defined | ||
|
|
||
| - no code here is executed, just type checked | ||
| - `# flags: ` indicates which flags to use for this unit test |
There was a problem hiding this comment.
Mention that this is optional.
test-data/unit/README.md
Outdated
| with text "abc..." | ||
| - note a space after `E:` and `flags:` | ||
| - lines without `# E: ` should cause no type check errors | ||
| - optional `[builtins]` tells the type checker to use stubs from the indicated |
There was a problem hiding this comment.
Maybe make it a bit more explicit, such as [builtins fixtures/...].
test-data/unit/README.md
Outdated
| `test-data/unit/lib-stub`. | ||
|
|
||
| - Individual test cases can override the stubs by using `[builtins | ||
| fixtures/foo.py]`; this targets files in `test-data/unit/fixtures`; feel |
There was a problem hiding this comment.
Use .pyi extension, not .py.
| (`mypy/test/testpythoneval.py`, `test-data/unit/pythoneval.test`). These | ||
| type check programs and run them. Unlike the unit tests, these use the | ||
| full builtins and library stubs instead of minimal ones. Run them using | ||
| `runtests.py testpythoneval`. |
There was a problem hiding this comment.
Maybe add mention of the runtests.py -j option and the pytest -n option for running multiple tests in parallel. These can speed up tests a lot, especially if you have many cores.
test-data/unit/README.md
Outdated
|
|
||
| `runtests.py` by default runs tests in parallel using as many processes as | ||
| there are logical cores the `runtests.py` process is allowed to use (on | ||
| Windows this information isn't available, so 2 processes are used by |
There was a problem hiding this comment.
This doesn't seem to work on macOS either, so maybe update the comment to read as "on some platforms this information ...".
test-data/unit/README.md
Outdated
| may provide the necessary dependencies. | ||
|
|
||
| To use the feature, pass e.g. `--txt-report "$(mktemp -d)"`. | ||
|
|
There was a problem hiding this comment.
Extra empty lines at the end of the file.
|
Thanks! This is going to very useful for new contributors. |
Add test-data/unit/README.md: move info from wiki, add more stuff