Add the matrix strategy to workflow to make integration tests faster #137
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nhinsch
reviewed
Apr 28, 2021
scripts/run_integration_tests.sh
Outdated
| run_id=$(xxd -l 4 -c 4 -p < /dev/random) | ||
|
|
||
| # Always remove the stack before exiting, no matter what | ||
| # Always remove the stacks before exiting, no matter what |
There was a problem hiding this comment.
Nit: I see you changed stack -> stacks here, was that intentional? Isn't this still going to deploy one stack at a time?
There was a problem hiding this comment.
I guess it should now be stack(s)
nhinsch
reviewed
Apr 28, 2021
.github/workflows/build.yml
Outdated
|
|
||
| strategy: | ||
| matrix: | ||
| runtime-param: [27, 36, 37, 38] |
There was a problem hiding this comment.
Nit: Can we use strings with decimals, i.e. "2.7" and "3.6", for these params? I always do a double take when I read "Python 37". Also, that will be more consistent with our unit test task. If it's not possible then no big deal.
nhinsch
approved these changes
Apr 28, 2021
nhinsch
reviewed
Apr 28, 2021
scripts/run_integration_tests.sh
Outdated
| PARAMETERS_SETS=(python${RUNTIME_PARAM}) | ||
| BUILD_LAYER_VERSION=python$RUNTIME_PARAM[1] | ||
| RUNTIME_PARAM_NO_DOT=$(echo $RUNTIME_PARAM | sed 's/\.//') | ||
| echo "Python version is specified: $RUNTIME_PARAM_NO_DOT" |
There was a problem hiding this comment.
Nit: I think we would probably want to log the version number with the dot here, right? Python version is specified: 3.7 rather than Python version is specified: 37
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR makes the integration testing step faster by using the matrix strategy, it will run tests in parallel.
Current average workflow time for the integration tests step : around 10 minutes
With this PR, the new time is about 4minutes
Motivation
Increase productivity by reducing waiting time
Testing Guidelines
Codebase was not change but a refactoring for integration testing has been made. This should not break the integration tests and should be fully backward compatible
Types of Changes
Check all that apply