Skip to content

Conversation

stephenyan1231
Copy link
Contributor

New features

Implement the following transforms for video clips

  • RandomCropVideo
  • RandomResizedCropVideo
  • CenterCropVideo
  • NormalizeVideo
  • ToTensorVideo
  • RandomHorizontalFlipVideo

Unit test

  • affected image tranfsorms
    • test/test_transforms.py
  • new unit test of video transforms
    • test/test_transforms_video

zyan3 and others added 27 commits September 6, 2019 22:51
* Register torchvision ops

* install ORT only with python 3

* remane lib + address other comments

* fix lint

* fix lib copy

* find file with pattern instead of suffix

* use relative path

* revert rename and use imp to find lib

* fix typo
* [WIP] Add CircleCI for CI

* Make jobs only run on master

* Add initial CI

* [wip] testing if works

* Trying some basic GPU tests

* [WIP] maybe it will work?

* [WIP] One more try

* Pin versions

* Simplify and reuse

* Fix

* [WIP] testing windows

* [WIP] testing windows

* Try windows

* Try Windows

* Try windows

* Try windows

* Try windows

* Try windows

* Try windows

* Try windows

* Windows speedup

* Windows multicores

* Add parallel flags for Windows

* Skip some tests on Windows

* Sync config.yml and config.yml.in

* Regenerate

* Run all tests

* Limit python and cuda version for finding pytorch

* Skip darwin for previous check

* Add description

* Fix logic

* Remove space

* Add CUDA test back

* Add back .travis.yml for now and remove duplicate test

* Add newline
* fix url

* update comment
* Revert "Revert "Register Torchvision Ops as Cutom Ops (pytorch#1267)" (pytorch#1316)"

This reverts commit fe234fc.

* Make import of C++ extensions lazy

* define python initialization functions for extension

* Fix lint
* Make custom ops differentiable

and replace autograd.Function. Use ops unconditionally.

We may consider removing the extension functions in a follow-up.

The code-path is tested by the exisitng tests for differentiability.

* add scripting gradchecks tests and use intlist

* fix implicit tuple conversion for gcc-5

* fix merge
* Adds optional fill colour to rotate

* bug fix
use flake8 with Python 3 on .circleci directory
* Support Exporting GeneralizedRCNNTransform

* refactor code to address comments

* update tests

* address comments

* revert min_size to test CI

* re-revert min_size
* Set block style serialization in yaml

* Add new checkout command

* Put on a different branch name
* Windows build scripts

Add testing yaml

* Change urls

* Test MS agent

* Switch base image

* Use fb links
* Fix Windows CI

* Use correct version
* Make AnchorGenerator support half precision

* Add test for fasterrcnn with double

* convert gt_boxes to right dtype
* Add Hub tests in torchvision

* Run test_hub in its own interpreter
* Remove C++ extensions in favor of custom ops

* Remove unused custom_ops.cpp file

* Rename _custom_ops.py

* Reorganize functions

* Minor improvements and fixes

* Fix lint

* Fully scriptable ops

* Import types used by annotations
@codecov-io
Copy link

codecov-io commented Sep 19, 2019

Codecov Report

❗ No coverage uploaded for pull request base (master@04f70c1). Click here to learn what that means.
The diff coverage is 78.24%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master    #1306   +/-   ##
=========================================
  Coverage          ?   65.98%           
=========================================
  Files             ?       77           
  Lines             ?     5932           
  Branches          ?      900           
=========================================
  Hits              ?     3914           
  Misses            ?     1746           
  Partials          ?      272
Impacted Files Coverage Δ
torchvision/models/detection/roi_heads.py 55.77% <0%> (ø)
torchvision/datasets/voc.py 21.64% <0%> (ø)
torchvision/datasets/mnist.py 51.7% <100%> (ø)
torchvision/__init__.py 76.19% <100%> (ø)
torchvision/models/resnet.py 88.48% <100%> (ø)
torchvision/ops/_register_onnx_ops.py 100% <100%> (ø)
torchvision/ops/__init__.py 100% <100%> (ø)
torchvision/ops/roi_pool.py 72% <100%> (ø)
torchvision/ops/roi_align.py 69.23% <100%> (ø)
torchvision/models/detection/rpn.py 79.32% <100%> (ø)
... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04f70c1...07c4393. Read the comment docs.

@stephenyan1231
Copy link
Contributor Author

Current PR is replaced with #1353

facebook-github-bot pushed a commit to facebookresearch/ClassyVision that referenced this pull request Oct 17, 2019
Summary:
Pull Request resolved: #53

- We add video transforms in TorchVision: pytorch/vision#1306
- In ClassyVision, we add default transforms for training / test stage. Alternatively, user can also explicitly provide transform config in json config input. See an example in the unit test.
-  Video data transforms supports audio modality in the video dataset.
-  Compared with image transforms which only returns a torch.Tensor, video transforms return a dict where key is the modality name (e.g. {"video", "audio"}) and value is a torch.Tensor for the modality data.

Reviewed By: taylorgordon20

Differential Revision: D16999453

fbshipit-source-id: 112b66a3965cba4201bbb12c99f3fdd2f1fce86f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants