feat: Add support for `require_full_compilation` in Dynamo #2138

gs-olive · 2023-07-25T00:20:52Z

Description

When a graph is fully supported, we can ignore the minimum block size argument, which is primarily helpful in reducing segmentation. If the minimum block size is above the number of total operators in the graph, and we support all of those, the whole graph will run in Torch regardless. As a result, we can exempt fully supported graphs from the min block size requirement

Fixes #2131

Type of change

New feature (non-breaking change which adds functionality)

Checklist:

[ x ] My code follows the style guidelines of this project (You can use the linters)
[ x ] I have performed a self-review of my own code
[ x ] I have commented my code, particularly in hard-to-understand areas and hacks
[ x ] I have made corresponding changes to the documentation
[ - ] I have added tests to verify my fix or my feature
- CI verification
[ x ] New and existing unit tests pass locally with my changes
[ x ] I have added the relevant labels to my PR in so that relevant reviewers are notified

py/torch_tensorrt/dynamo/lowering/_partition.py

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

py/torch_tensorrt/dynamo/lowering/_partition.py

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

tests/py/dynamo/shared_utilities/test_partitioning.py

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

tests/py/dynamo/models/__init__.py

github-actions

Code conforms to C++ style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

- When a graph is fully supported, we can ignore the minimum block size argument, which is primarily helpful in reducing segmentation. If the minimum block size is above the number of total operators in the graph, and we support all of those, the whole graph will run in Torch regardless. As a result, we can exempt fully supported graphs from the min block size requirement - Alternatively, if preferable, we can display a warning in such a case, but still respect the minimum block size argument refactor: Add require_full_compilation in Dynamo - Add support for full compilation compilation argument in Dynamo paths feat: Disable require_full_compilation in torch compile

github-actions

Code conforms to Python style guidelines

github-actions

Code conforms to C++ style guidelines

gs-olive added component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths Story: Export/Compile Unification Issues relating to unification of Dynamo compile/export paths labels Jul 25, 2023

gs-olive self-assigned this Jul 25, 2023

facebook-github-bot added the cla signed label Jul 25, 2023

github-actions bot added the component: api [Python] Issues re: Python API label Jul 25, 2023

github-actions bot requested a review from narendasan July 25, 2023 00:21

gs-olive commented Jul 25, 2023

View reviewed changes

py/torch_tensorrt/dynamo/lowering/_partition.py Show resolved Hide resolved

github-actions bot approved these changes Jul 25, 2023

View reviewed changes

gs-olive force-pushed the min_block_size_exemption branch from f59658b to ee1f65d Compare July 25, 2023 17:32

github-actions bot added the component: tests Issues re: Tests label Jul 25, 2023

github-actions bot approved these changes Jul 25, 2023

View reviewed changes

gs-olive force-pushed the min_block_size_exemption branch from ee1f65d to 220ee6a Compare July 26, 2023 01:52

github-actions bot approved these changes Jul 26, 2023

View reviewed changes

gs-olive changed the title ~~feat: Add support for exempting fully-supported graphs from min_block_size~~ feat: Add support for require_full_compilation in Dynamo Jul 26, 2023

gs-olive commented Aug 1, 2023

View reviewed changes

py/torch_tensorrt/dynamo/lowering/_partition.py Show resolved Hide resolved

gs-olive force-pushed the min_block_size_exemption branch from 220ee6a to b04c8a9 Compare August 2, 2023 21:13

github-actions bot added component: lowering Issues re: The lowering / preprocessing passes component: torch_compile labels Aug 2, 2023

github-actions bot approved these changes Aug 2, 2023

View reviewed changes

gs-olive force-pushed the min_block_size_exemption branch from b04c8a9 to 413e9f9 Compare August 9, 2023 06:21

github-actions bot approved these changes Aug 9, 2023

View reviewed changes

gs-olive force-pushed the min_block_size_exemption branch from 413e9f9 to 02ecf05 Compare August 15, 2023 22:40

github-actions bot approved these changes Aug 15, 2023

View reviewed changes

gs-olive force-pushed the min_block_size_exemption branch from 02ecf05 to 3dc4af0 Compare August 16, 2023 01:01

gs-olive force-pushed the min_block_size_exemption branch from b620a1f to c1129e6 Compare August 17, 2023 23:41

github-actions bot approved these changes Aug 17, 2023

View reviewed changes

gs-olive commented Aug 18, 2023

View reviewed changes

tests/py/dynamo/shared_utilities/test_partitioning.py Show resolved Hide resolved

gs-olive force-pushed the min_block_size_exemption branch from c1129e6 to 9bd42d0 Compare August 18, 2023 21:46

github-actions bot approved these changes Aug 18, 2023

View reviewed changes

gs-olive force-pushed the min_block_size_exemption branch from 9bd42d0 to 81065ad Compare August 19, 2023 04:05

github-actions bot approved these changes Aug 19, 2023

View reviewed changes

gs-olive force-pushed the min_block_size_exemption branch from 81065ad to d9761a6 Compare August 22, 2023 16:12

github-actions bot approved these changes Aug 22, 2023

View reviewed changes

gs-olive commented Aug 22, 2023

View reviewed changes

tests/py/dynamo/models/__init__.py Show resolved Hide resolved

gs-olive force-pushed the min_block_size_exemption branch from d9761a6 to 0f3e460 Compare August 23, 2023 00:29

gs-olive changed the base branch from main to testing_reorg August 23, 2023 00:30

github-actions bot approved these changes Aug 23, 2023

View reviewed changes

gs-olive force-pushed the testing_reorg branch 2 times, most recently from 37b963a to c8c906e Compare August 25, 2023 20:44

gs-olive force-pushed the min_block_size_exemption branch from 0f3e460 to c835e6b Compare August 28, 2023 02:41

github-actions bot approved these changes Aug 28, 2023

View reviewed changes

Base automatically changed from testing_reorg to main August 28, 2023 16:00

gs-olive force-pushed the min_block_size_exemption branch from c835e6b to b5ed900 Compare August 28, 2023 17:25

github-actions bot approved these changes Aug 28, 2023

View reviewed changes

gs-olive merged commit 8efbee9 into main Aug 29, 2023

gs-olive deleted the min_block_size_exemption branch August 29, 2023 16:44

feat: Add support for require_full_compilation in Dynamo #2138

feat: Add support for require_full_compilation in Dynamo #2138

Uh oh!

Conversation

gs-olive commented Jul 25, 2023

Description

Type of change

Checklist:

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

feat: Add support for `require_full_compilation` in Dynamo #2138

feat: Add support for `require_full_compilation` in Dynamo #2138