Skip to content

Lazy tests: parse using regex first #5459

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 26, 2018
Merged

Conversation

elazarg
Copy link
Contributor

@elazarg elazarg commented Aug 12, 2018

Fix #1073: parse tests on collection only enough to find the name, so small number of tests run faster

On my machine, pytest -n0 -k testAttrsSimple takes at least 2.24 seconds to finish on master, and at most 0.95 seconds to finish with this PR.

  • 'skip-cache' is changed to 'only_when_nocache' and similarly 'skip-nocache'
  • I have replace while loops with "if True" to make the diff simpler. Further cleanup and optimizations are also possible

@elazarg elazarg force-pushed the lazy-tests-regex branch 2 times, most recently from a0956c0 to 999db69 Compare August 13, 2018 03:46
- change 'skip-cache' to 'only_when_nocache' and similarly
'skip-nocache'
@msullivan msullivan self-requested a review September 7, 2018 06:42
Copy link
Collaborator

@msullivan msullivan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the big delay in reviewing this.

The speedups for running single tests will be really nice and I am happy with the approach here.

Could you go ahead and get rid of the if Trues and then we can merge this?

@elazarg
Copy link
Contributor Author

elazarg commented Sep 25, 2018

Additional refactoring and cleanup:

  • rename variables to be consistent and clear (e.g. test_items instead of p)
  • extract variables and switch to for-loop
  • combine the out cases, removing repetition
  • rechecked/stale cases reformatted so the repetition is obvious
  • fail faster in case out_section_missing, with a clearer error message

@elazarg
Copy link
Contributor Author

elazarg commented Sep 25, 2018

Linux 3.4 / 3.5.1 seem to fail spuriously - "Unable to download 3.5.1 archive"?

(The windows tests fail due to path separator on the expected output being reversed when it shouldn't)

@msullivan
Copy link
Collaborator

I've restarted the linux builds

@msullivan msullivan merged commit 03588e7 into python:master Sep 26, 2018
@msullivan
Copy link
Collaborator

Thank you!

@gvanrossum
Copy link
Member

Since this PR landed, pytest -v mypy/test/testcheck.py prints a stream of messages like this:

[location] PASSED                   [  0%]
[location] PASSED                   [  0%]
[location] PASSED                   [  0%]

Previously it would print the test name instead of the string [location], e.g.:

mypy/test/testcheck.py::TypeCheckSuite::testEmptyFile <- /Users/guido/src/mypy-master/test-data/unit/check-basic.test PASSED                                              [  0%]
mypy/test/testcheck.py::TypeCheckSuite::testAssignmentAndVarDef <- /Users/guido/src/mypy-master/test-data/unit/check-basic.test PASSED                                    [  0%]
mypy/test/testcheck.py::TypeCheckSuite::testConstructionAndAssignment <- /Users/guido/src/mypy-master/test-data/unit/check-basic.test PASSED                              [  0%]

Methinks something went wrong here. @elazarg do you have a hunch as to what caused this? (I feel I've done enough research by bisecting it to this PR.)

@elazarg
Copy link
Contributor Author

elazarg commented Oct 4, 2018

Of course. Sorry!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants