[jobs] testing: reduce test flakiness #3278
Conversation
googlebot
added
the
cla: yes
fixes GoogleCloudPlatform#2945 fixes GoogleCloudPlatform#2818 There were 8 tests which has `time.sleep(10)` for waiting data propagation or whatever. I chose to use `@eventually_consistent.call` instead of sleep in those tests. Bonus point is now the tests are significantly faster because it doesn't always wait for 10 seconds. The downside is it will take a long time on failure. We may want to specify the retry count on the eventualy consistent decorator once the following issue is fixed: GoogleCloudPlatform/python-repo-tools#25
| @@ -1,3 +1,4 @@ | |||
| pytest==5.3.2 | |||
| gcp-devrel-py-tools==0.0.15 | |||
| google-cloud-core | |||
There was a problem hiding this comment.
How is google-cloud-core being used? I'm not seeing it anywhere. I could easily be missing something though
There was a problem hiding this comment.
It is used by gcp-devrel-py-tools, which has a broken dependency.
See: GoogleCloudPlatform/python-repo-tools#24
I'll fix them with upcoming PR.
gguuss
left a comment
gguuss
left a comment
There was a problem hiding this comment.
Only thing that jumped out at me was the use of gcp_devrel tools, I believe the python-samples-owners team is working to remove this library from our tests.
| def test_commute_search_sample(capsys): | ||
| import commute_search_sample | ||
| import re | ||
| from gcp_devrel.testing import eventually_consistent |
There was a problem hiding this comment.
I believe we're discouraging use of gcp_devrel and instead preferring flaky or retry. Is there an advantage to eventually_consistent?
There was a problem hiding this comment.
eventually_consistent is a thin wrapper around retrying, specialized to eventually consistent stuff. I'm not sticking to use gcp_devrel, but the code is more readable and descriptive. I'm more than happy to maintain that library, so there's no worry of it being not maintained.
If you strongly feel we should use retrying, I can do that though.
|
Thanks! When we decide to use |
5 participants
fixes #2945
fixes #2818
There were 8 tests which has
time.sleep(10)for waiting datapropagation or whatever. I chose to use
@eventually_consistent.callinstead of sleep in those tests.
Bonus point is now the tests are significantly faster because it doesn't
always wait for 10 seconds.
The downside is it will take a long time on failure. We may want to
specify the retry count on the eventualy consistent decorator once the
following issue is fixed:
GoogleCloudPlatform/python-repo-tools#25