-
Notifications
You must be signed in to change notification settings - Fork 15
Add performance tests to compare crud and vshard #251
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
DifferentialOrange
merged 2 commits into
master
from
DifferentialOrange/gh-225-vshard-performance
Mar 4, 2022
Merged
Add performance tests to compare crud and vshard #251
DifferentialOrange
merged 2 commits into
master
from
DifferentialOrange/gh-225-vshard-performance
Mar 4, 2022
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
e976a4a
to
580ba0a
Compare
23c6445
to
3332a01
Compare
580ba0a
to
21420dd
Compare
1 task
5deebbd
to
dc06b81
Compare
10733cb
to
859a624
Compare
bea37df
to
afec422
Compare
23cf464
to
8e6e5f9
Compare
df98d90
to
b2dd54c
Compare
b2dd54c
to
c34491e
Compare
8c054db
to
0ed9215
Compare
Base automatically changed from
DifferentialOrange/gh-224-operation-stats
to
master
February 25, 2022 09:41
b0b9690
to
920523e
Compare
Totktonada
reviewed
Mar 3, 2022
Totktonada
reviewed
Mar 3, 2022
Totktonada
reviewed
Mar 3, 2022
Totktonada
reviewed
Mar 3, 2022
Totktonada
reviewed
Mar 3, 2022
Totktonada
reviewed
Mar 3, 2022
Totktonada
reviewed
Mar 3, 2022
Totktonada
reviewed
Mar 3, 2022
Totktonada
reviewed
Mar 3, 2022
Totktonada
approved these changes
Mar 3, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
Quite interesting results, thanks!
I left some minor comments. Feel free to fix or go ahead.
Fix replicaset alias in test cluster. Disable checks for perf tests. Return tuples from pairs perf case. Fix running separate tests. After this patch, developer can run luatest with -p flag for perf tests to get results only for requested tests. Example: ``` PERF_MODE_ON=true ./.rocks/bin/luatest -c -p perf.test_crud_pairs_gt_with_stats_disabled ```
920523e
to
fc249e9
Compare
This patch adds new cases for performance tests: select for equal conditions for primary and secondary indexes (including known bucket_id case from #220), and adds corresponding vshard test cases to compare performance. Comparison may be not exactly precise since approaches to vshard stored procedures may vary, but it should at least estimate basic differences. Test run on HP ProBook 440 G7 i7/16Gb/256SSD shows that CRUD module is 3-6 times slower than vshard calls for prepared functions for select and 1.6 times slower for insert. Closes #225
fc249e9
to
3c53164
Compare
Open
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch adds new cases for performance tests: select for equal
conditions for primary and secondary indexes (including known bucket_id
case from #220), and adds corresponding vshard test cases to compare
performance. Comparison may be not exactly precise since vshard test
functions use naive mergers, but should at least estimate basic
differences.
Test run on HP ProBook 440 G7 i7/16Gb/256SSD shows that CRUD module is
3-6 times slower than vshard calls for prepared functions for select and
1.6 times slower for insert.
HP ProBook 440 G7 i7/16Gb/256SSD
SUCCESS REQUESTS PER SECOND
(The higher the better)
AVERAGE CALL TIME
(The lower the better)
MAX CALL TIME
(The lower the better)
CI Runner
See https://github.com/tarantool/crud/runs/5421561722?check_suite_focus=true
Several minor fixes also was introduced.
Fix replicaset alias in test cluster. Disable checks for perf tests.
Return tuples from pairs perf case.
Fix running separate tests. After this patch, developer can run luatest
with -p flag for perf tests to get results only for requested tests.
Example:
Closes #225