-
-
Notifications
You must be signed in to change notification settings - Fork 11.8k
[Tests] fix initialization of kv hash in tests #24273
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Tests] fix initialization of kv hash in tests #24273
Conversation
otherwise they implicitly require to run sequentially Signed-off-by: Mickael Seznec <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request addresses an issue with test isolation by introducing a with_init_none_hash pytest fixture to ensure the KV cache hash is initialized before tests that require it. This is a good improvement for test stability and allows for parallel execution. The fixture is applied to many tests across the suite. My review identifies a couple of minor cases of redundancy where init_none_hash is called explicitly within tests that also use the new fixture. Removing these redundant calls will clean up the code.
Signed-off-by: Mickael Seznec <[email protected]>
|
cc @vMaroon |
tests/conftest.py
Outdated
|
|
||
|
|
||
| @pytest.fixture | ||
| def with_init_none_hash(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This definitely is a step towards cleaner testing, though it may look out of place when passed as test argument without much context.
Perhaps this can be taken a step further by using the @pytest.fixture(autouse=True) decorator and either:
- Implicitly calling
init_none_hashwhere needed:
pytest.fixture(autouse=True)
def _auto_init_hash_fn(request):
if "hash_fn" in request.fixturenames:
init_none_hash(request.getfixturevalue("hash_fn"))
- Explicit dependency
@pytest.fixture
def initialized_hash_fn(hash_fn):
"""Fixture that automatically initializes the hash function."""
init_none_hash(hash_fn)
return hash_fn
# Update relevant tests to use the above fixture, e.g.:
@pytest.mark.parametrize("hash_fn", [sha256, sha256_cbor_64bit, hash])
def test_hash_block_tokens(initialized_hash_fn):
# Replace the manual init_none_hash call
hash_fn = initialized_hash_fn
# ...
@mickaelseznec What do you think?
Both can optionally be enhanced by capturing and restoring global state by utilizing yield.
I think either is good, personal preference is to the first. I assume you tagged me because I introduced the manual inits, but maintainer input is required here (cc @heheda12345).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for the feedback! The first proposition looks good, let me update my PR :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can keep the thing simple. For example, auto-apply with_init_none_hash in test_prefix_caching.py and call init_none_hash manually in other tests if needed. This won't be too confusing as most tests require it is related to hash.
I don't what it to be a autouse by default because then CI can't detect problems due to un-initialized non-hash.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the nice ideas, I think I've come up with nice in-between:
- autouse fixture for the two relevant files only
It has some code dup, but it's less intrusive (and probably easier to understand to read) than putting the fixture in conftest.py.
Let me know if that sounds good to you :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah make sense to me.
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Mickael Seznec <[email protected]>
d56a6b3 to
3097d6d
Compare
Signed-off-by: Mickael Seznec <[email protected]>
heheda12345
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Signed-off-by: Mickael Seznec <[email protected]>
heheda12345
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Signed-off-by: Mickael Seznec <[email protected]>
Signed-off-by: bbartels <[email protected]> [gpt-oss] Add IncompleteDetails to ResponsesRepsonse (vllm-project#24561) Signed-off-by: Andrew Xia <[email protected]> [gpt-oss][1a] create_responses stream outputs BaseModel type, api server is SSE still (vllm-project#24759) Signed-off-by: Andrew Xia <[email protected]> [Performance] Remove redundant clone() calls in cutlass_mla (vllm-project#24891) [Bug] Fix Cutlass Scaled MM Compilation Error (vllm-project#24887) Signed-off-by: yewentao256 <[email protected]> [ci] fix wheel names for arm wheels (vllm-project#24898) Signed-off-by: simon-mo <[email protected]> [Tests] fix initialization of kv hash in tests (vllm-project#24273) Signed-off-by: Mickael Seznec <[email protected]> [Compile] Fix noop_elimination pass and add tests for noop_elimination (vllm-project#24880) Signed-off-by: zjy0516 <[email protected]> Propagate entire tokens to connector for resumed preemptions Signed-off-by: Qier Li <[email protected]> Fix pre-commit Signed-off-by: Qier Li <[email protected]> Rename field and nullify empty lists Signed-off-by: Qier Li <[email protected]> Update vllm/v1/core/sched/scheduler.py Co-authored-by: Nick Hill <[email protected]> Signed-off-by: Qier Li <[email protected]> Add unit test for preemption resumption Signed-off-by: Qier Li <[email protected]>
Signed-off-by: Mickael Seznec <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
Signed-off-by: Mickael Seznec <[email protected]>
Signed-off-by: Mickael Seznec <[email protected]> Signed-off-by: xuebwang-amd <[email protected]>
Purpose
The tests pass, only because they run sequentially and the first do the necessary init. Now we can run them in any order (and parallel).
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.