Skip to content

Conversation

Charlie-XIAO
Copy link
Contributor

@Charlie-XIAO Charlie-XIAO commented Jun 28, 2023

In this PR, I added a pre-commit hook to check that pandas._libs.lib.NoDefault is used only for typing. This is invoked by #53877. Before that PR, this new hook correctly catches all misused NoDefault:

(pandas-dev) D:\>pre-commit run unwanted-patterns-nodefault-not-used-for-typing --all-files
Check for pandas._libs.lib.NoDefault not used for typing.......................Failed   
- hook id: unwanted-patterns-nodefault-not-used-for-typing
- exit code: 1

pandas/tests/reshape/test_pivot_multilevel.py:38:NoDefault is not used for typing       
pandas/tests/reshape/test_pivot_multilevel.py:75:NoDefault is not used for typing       
pandas/core/frame.py:8831:NoDefault is not used for typing
pandas/core/frame.py:8831:NoDefault is not used for typing
pandas/core/reshape/pivot.py:514:NoDefault is not used for typing
pandas/core/reshape/pivot.py:515:NoDefault is not used for typing
pandas/core/reshape/pivot.py:529:NoDefault is not used for typing
pandas/core/reshape/pivot.py:525:NoDefault is not used for typing
pandas/core/reshape/pivot.py:530:NoDefault is not used for typing
pandas/core/reshape/pivot.py:535:NoDefault is not used for typing
pandas/core/reshape/pivot.py:542:NoDefault is not used for typing
pandas/core/reshape/pivot.py:572:NoDefault is not used for typing

After that PR, the check passes normally:

(pandas-dev) D:\>pre-commit run unwanted-patterns-nodefault-not-used-for-typing --all-files
Check for pandas._libs.lib.NoDefault not used for typing.......................Passed

In fact I don't work with ASTs very often so not sure if this check covers all possibilties. Please let me know if any modification is needed. (The runtime is a bit long but seems acceptable, approximately 10s to check all files.)

@MarcoGorelli MarcoGorelli added Code Style Code style, linting, code_checks Typing type annotations, mypy/pyright type checking labels Jun 28, 2023
@MarcoGorelli MarcoGorelli added this to the 2.1 milestone Jun 28, 2023
@MarcoGorelli
Copy link
Member

nice idea

I think you're repeatedly walking up and down the same paths - do you want to check scripts/no_bool_in_generic.py? that one checks for annotations, maybe there's some logic there that can be reused?

@Charlie-XIAO
Copy link
Contributor Author

Charlie-XIAO commented Jun 28, 2023

Improved from approx. 10s for all files to approx. 4s @MarcoGorelli Thanks for your suggestion!

@MarcoGorelli
Copy link
Member

Well done!

Would appreciate it if we could add a tiny little test case to scripts/tests/test_validate_unwanted_patterns.py too

Other than that, looks good!

@Charlie-XIAO
Copy link
Contributor Author

Charlie-XIAO commented Jun 28, 2023

Didn't notice that there are tests for linting scripts, thanks for the reminder @MarcoGorelli. Added some simple tests now.

Copy link
Member

@MarcoGorelli MarcoGorelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, great work, looks good to me on green

@mroeschke mroeschke merged commit 69acdb2 into pandas-dev:main Jun 29, 2023
@mroeschke
Copy link
Member

Nice thanks @Charlie-XIAO

Daquisu pushed a commit to Daquisu/pandas that referenced this pull request Jul 8, 2023
…andas-dev#53901)

* CI add liniting to check NoDefault used only for typing

* minor modification

* reduce cost

* tests added for the new linting check

* types_or -> types because only python

* rephrase pre-commit hook name

* rephrase more

* fix failing tests:

* retrigger checks

* retrigger checks
@Charlie-XIAO Charlie-XIAO deleted the lint-nodefault-only-typing branch September 23, 2023 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Code Style Code style, linting, code_checks Typing type annotations, mypy/pyright type checking
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants