Decrease performance tests duration in stability testing #233
Labels
code health
Improve code readability, simplify maintenance and so on
Milestone
Comments
Totktonada
added
the
code health
DifferentialOrange
added a commit
that referenced
this issue
Dec 15, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also adds new cases to performance test to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers. Closes #224, #233
3 tasks
DifferentialOrange
added a commit
that referenced
this issue
Dec 15, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also adds new cases to performance test to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 15, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also adds new cases to performance test to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 15, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also adds new cases to performance test to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 15, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also adds new cases to performance test to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 15, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also adds new cases to performance test to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 17, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 20, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 20, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 22, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 22, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 22, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Dec 22, 2021
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Jan 18, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Jan 28, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Jan 29, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Jan 29, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Jan 29, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-7% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Jan 31, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-7% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Jan 31, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-7% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 4, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-7% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 4, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-7% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 17, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-7% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 18, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-7% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 18, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 18, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 18, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 18, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 18, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 18, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 21, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 21, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 21, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 21, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 21, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 22, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 22, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 22, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 22, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 24, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 24, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 24, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 24, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 24, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 24, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 25, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 25, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 25, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
DifferentialOrange
added a commit
that referenced
this issue
Feb 25, 2022
Before this patch, performance tests ran together with unit and integration with `--coverage` flag. Coverage analysis cropped the result of performance tests to 10-15 times. For metrics integration it resulted in timeout errors and drop of performance which is not reproduces with coverage disabled. Moreover, before this patch log capture was disabled and performance tests did not displayed any results after run. Now performance tests also run is separate CI job. After this patch, `make -C build coverage` will run lightweight version of performance test. `make -C build performance` will run real performance tests. You can paste output table to GitHub [1]. This path also reworks current performance test. It adds new cases to compare module performance with or without statistics, statistic wrappers and compare different metrics drivers and reports new info: average call time and max call time. Performance test result: overhead is 3-10% in case of `local` driver and 5-15% in case of `metrics` driver, up to 20% for `metrics` with quantiles. Based on several runs on HP ProBook 440 G7 i7/16Gb/256SSD. 1. https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables Closes #233, follows up #224
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
There is no sense to spin for 30 seconds on them in stability testing. I would run them for a long duration only when a particular environment variable is set, say
LUATEST_PERFTEST=1. In tarantool/luatest#186 we can set this variable from luatest itself, when perf. test mode is requested.The text was updated successfully, but these errors were encountered: