-
Notifications
You must be signed in to change notification settings - Fork 64
Mark slow tests #968
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mark slow tests #968
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding this!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM but let's mark the webm tests only, as you originally suggested. I shared an example below on how to do that
test/test_ops.py
Outdated
@pytest.mark.slow | ||
@pytest.mark.parametrize("format", ("mov", "mp4", "mkv", "webm")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can "mark" individual entries with arbitrary marks:
@pytest.mark.slow | |
@pytest.mark.parametrize("format", ("mov", "mp4", "mkv", "webm")) | |
@pytest.mark.parametrize( | |
"format", ("mov", "mp4", "mkv", pytest.param("webm", marks=pytest.mark.slow)) | |
) |
This is also how we handle our automatic "needs_cuda" mark:
Lines 29 to 34 in b9cf8d1
def all_supported_devices(): | |
return ( | |
"cpu", | |
pytest.param("cuda", marks=pytest.mark.needs_cuda), | |
pytest.param("cuda:0:beta", marks=pytest.mark.needs_cuda), | |
) |
test/test_ops.py
Outdated
@pytest.mark.slow | ||
@pytest.mark.skipif(in_fbcode(), reason="ffmpeg CLI not available") | ||
@pytest.mark.parametrize( | ||
"format", ("mov", "mp4", "avi", "mkv", "webm", "flv", "gif") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here
Both
test_video_encoder_round_trip()
andtest_video_encoder_against_ffmpeg_cli()
are taking around 30 seconds or more on my dev machine for webm. If we could just say that the webm option is slow, that would be better, but I think this is okay for now.This is relevant for when we just say
pytest test
, it will default to not running the slow tests. See ourpyproject.toml
:torchcodec/pyproject.toml
Lines 43 to 51 in b9cf8d1