Skip to content

Single script to run PR checks locally #8686

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Avasam opened this issue Sep 5, 2022 · 3 comments · Fixed by #8798
Closed

Single script to run PR checks locally #8686

Avasam opened this issue Sep 5, 2022 · 3 comments · Fixed by #8798
Labels
project: infrastructure typeshed build, test, documentation, or distribution related

Comments

@Avasam
Copy link
Collaborator

Avasam commented Sep 5, 2022

After running scripts/create_baseline_stubs.py, we are informed of the following

Suggested next steps:
1. Manually review the generated stubs in {stub_dir}
2. Run "MYPYPATH={stub_dir} python3 -m mypy.stubtest {package}" to check the stubs against runtime
3. Run "mypy {stub_dir}" to check for errors
4. Run "black {stub_dir}" and "isort {stub_dir}" (if you\'ve made code changes)
5. Run "flake8 {stub_dir}" to check for e.g. unused imports
6. Commit the changes on a new branch and create a typeshed PR

There's a few issues with running those commands as such:

  1. Setting the MYPYPATH environment variable and running a script afterward is done differently per shell (think of powershell for example, where its $Env:MYPYPATH="{stub_dir}"; python3 -m mypy.stubtest {package})
  2. These will ignore some configurations such as METADATA.toml, stubtest_allowlist.txt, etc.
  3. Does not include other checks defined in https://github.com/python/typeshed/blob/master/tests/README.md
  4. It takes multiple commands to autofix and check everything
  5. Overall, the results will be different from what's on the CI

For those reasons, I suggest adding a new script under scripts that a dev who's contributing to typeshed can run locally to easily and completely check their stubs. The documentation in CONTRIBUTING.md and the output of scripts/create_baseline_stubs.py should then be updated accordingly.

@Akuli
Copy link
Collaborator

Akuli commented Sep 5, 2022

I would prefer:

1. Manually review the generated stubs in {stub_dir}
2. Run tests locally if you want (see CONTRIBUTING.md)
3. Commit the changes on a new branch and create a typeshed PR

The CI will check everything anyway, and is set up so that you don't have to run anything locally. This would also be consistent with CONTRIBUTING.md.

@AlexWaygood AlexWaygood added the project: infrastructure typeshed build, test, documentation, or distribution related label Sep 5, 2022
@Avasam
Copy link
Collaborator Author

Avasam commented Sep 7, 2022

@Akuli I'm mainly asking this because of the constant back and forth this produces with pushing changes, waiting on the CI (which has additional overhead), opening the errors page and dropdowns, then searching for the error. I'd much rather be able to quickly run all autofixes and checks locally.

My editor being VSCode, I could just setup a task for this project to run everything with the click of a single button. (but that's only gonna be beneficial to me)

At the end of the day, it's not a major pain point, and most contributions will only include some changes in already compliant stubs. I just happened to be using quite a handful of popular untyped libraries of which most were not yet ready to include types directly.

@AlexWaygood
Copy link
Member

AlexWaygood commented Sep 7, 2022

I think we can maybe do both.

I agree with @Akuli that the language in create_baseline_stubs.py is definitely outdated — nowadays, we're pretty keen to emphasise that contributors shouldn't have to run any tests or formatting locally before submitting a PR.

But that doesn't mean that we can't also have a convenience script to e.g. run pycln/isort/black/flake8/mypy in quick succession, for contributors who want to run the tests locally first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
project: infrastructure typeshed build, test, documentation, or distribution related
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants