Skip to content

Reduce sample inputs for prototype transform kernels #6714

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Oct 6, 2022

Conversation

pmeier
Copy link
Collaborator

@pmeier pmeier commented Oct 6, 2022

Addresses 3. from #6622.

This PR achieves a significant reduction of the number of sample inputs. Only for the bounding box kernels, the number goes up slightly. After manual inspection, it became clear that our previous sample inputs were not sufficient.

kernel # sample inputs main # sample inputs PR
pad_image_tensor 2448 108
resize_image_tensor 960 73
pad_mask 864 12
center_crop_image_tensor 864 48
resize_mask 720 12
center_crop_mask 648 12
affine_image_tensor 624 180
ten_crop_image_tensor 480 60
rotate_image_tensor 432 84
crop_image_tensor 384 30
affine_mask 288 12
convert_color_space_image_tensor 288 168
crop_mask 288 12
gaussian_blur_image_tensor 288 36
affine_bounding_box 252 264
five_crop_image_tensor 240 30
pad_bounding_box 108 252
rotate_mask 72 12
resize_bounding_box 60 180

In total we cut the number of tests (and approximately the runtime) roughly to a third of what is was before. Plus, we even increase the coverage ever so slightly:

  • main:
    ---------- coverage: platform linux, python 3.7.12-final-0 -----------
    Name                                                              Stmts   Miss  Cover
    -------------------------------------------------------------------------------------
    torchvision/prototype/transforms/functional/__init__.py               8      0   100%
    torchvision/prototype/transforms/functional/_augment.py              18      1    94%
    torchvision/prototype/transforms/functional/_color.py                91      0   100%
    torchvision/prototype/transforms/functional/_deprecated.py           29     17    41%
    torchvision/prototype/transforms/functional/_geometry.py            539     32    94%
    torchvision/prototype/transforms/functional/_meta.py                137     28    80%
    torchvision/prototype/transforms/functional/_misc.py                 48      7    85%
    torchvision/prototype/transforms/functional/_type_conversion.py      31      7    77%
    -------------------------------------------------------------------------------------
    TOTAL                                                               901     92    90%
    
    ===== 66202 passed, 87703 skipped, 1792 xfailed, 1 warning in 173.32s (0:02:53) =====
    
  • PR:
    ---------- coverage: platform linux, python 3.7.12-final-0 -----------
    Name                                                              Stmts   Miss  Cover
    -------------------------------------------------------------------------------------
    torchvision/prototype/transforms/functional/__init__.py               8      0   100%
    torchvision/prototype/transforms/functional/_augment.py              18      1    94%
    torchvision/prototype/transforms/functional/_color.py                91      0   100%
    torchvision/prototype/transforms/functional/_deprecated.py           29     17    41%
    torchvision/prototype/transforms/functional/_geometry.py            539     31    94%
    torchvision/prototype/transforms/functional/_meta.py                137     25    82%
    torchvision/prototype/transforms/functional/_misc.py                 48      7    85%
    torchvision/prototype/transforms/functional/_type_conversion.py      31      7    77%
    -------------------------------------------------------------------------------------
    TOTAL                                                               901     88    90%
    
    ====== 21785 passed, 21954 skipped, 544 xfailed, 1 warning in 60.59s (0:01:00) ======
    

@@ -63,17 +63,40 @@ def sample_inputs(self, *feature_types, filter_metadata=True):
yield args_kwargs


def xfail_python_scalar_arg_jit(name, *, reason=None):
def xfail_jit_python_scalar_arg(name, *, reason=None):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a rename that moves the jit term to the front of the name to make it clear this is only a JIT issue.

Comment on lines +164 to +165
# TODO: check if this is a regression since it seems that should be supported if `int` is ok
xfail_jit_list_of_ints("fill"),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a few of these. I'm not sure if this is a regression of #6636. Will check and send a follow-up PR since this is one is only test changes.

@@ -755,12 +755,7 @@ def gaussian_blur(img: Tensor, kernel_size: List[int], sigma: List[float]) -> Te
kernel = _get_gaussian_kernel2d(kernel_size, sigma, dtype=dtype, device=img.device)
kernel = kernel.expand(img.shape[-3], 1, kernel.shape[0], kernel.shape[1])

img, need_cast, need_squeeze, out_dtype = _cast_squeeze_in(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a formatting change like #5880 that I saw while going through the kernels.

@@ -194,7 +225,12 @@ def fill_sequence_needs_broadcast(args_kwargs):
),
condition=lambda args_kwargs: fill_sequence_needs_broadcast(args_kwargs)
and args_kwargs.kwargs.get("padding_mode", "constant") == "constant",
)
),
xfail_jit_python_scalar_arg("padding"),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why all this xfail appeared in this PR and not before ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we didn't test scalar padding before

padding=[[1], [1, 1], [1, 1, 2, 2]],

Thus, while reducing the number of sample inputs now, the tests are actually more comprehensive.

Copy link
Collaborator

@vfdev-5 vfdev-5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @pmeier !

@pmeier pmeier merged commit 026991b into pytorch:main Oct 6, 2022
@pmeier pmeier deleted the reduce-sample-inputs branch October 6, 2022 13:43
facebook-github-bot pushed a commit that referenced this pull request Oct 17, 2022
Summary:
* pad_image_tensor

* pad_mask and pad_bounding_box

* resize_{image_tensor, mask, bounding_box}

* center_crop_{image_tensor, mask}

* {five, ten}_crop_image_tensor

* crop_{image_tensor, mask}

* convert_color_space_image_tensor

* affine_{image_tensor, mask, bounding_box}

* rotate_{image_tensor, mask}

* gaussian_blur_image_tensor

* cleanup

Reviewed By: NicolasHug

Differential Revision: D40427478

fbshipit-source-id: 4954886d7d0b8212ea92847b07092a8dfad809cb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants