fix/feat: Add support for 64bit Tensor inputs FX aten [9 / x] #2021
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
component: api [Python]
gs-olive
commented
Jun 20, 2023
Comment on lines
+8
to
+9
| from torch_tensorrt._Device import Device | ||
| from torch_tensorrt._enums import EngineCapability |
There was a problem hiding this comment.
These changes were required to avoid circular import errors in Python. Updates to #1983 could potentially fix this issue.
gs-olive
force-pushed
the
dynamo_compile_64bit_support
branch
from
76a4cdc to
6635bdc
Compare
gs-olive
force-pushed
the
fx_aten_64bit_support
branch
from
d533463 to
4e843a3
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
There are some changes that do not conform to Python style guidelines:
--- py/torch_tensorrt/dynamo/backend/utils.py 2023-06-21 01:08:42.236399 +0000
+++ py/torch_tensorrt/dynamo/backend/utils.py 2023-06-21 01:09:05.163004 +0000
@@ -73,10 +73,11 @@
"Invalid device provided. Supported options: torch.device | torch_tensorrt.Device"
)
return device
+
def parse_dynamo_kwargs(kwargs: Dict) -> CompilationSettings:
"""Parses the kwargs field of a Dynamo backend
Args:
kwargs: Keyword arguments dictionary provided to the backend
gs-olive
force-pushed
the
dynamo_compile_64bit_support
branch
from
6635bdc to
7548ad2
Compare
gs-olive
force-pushed
the
fx_aten_64bit_support
branch
from
4e843a3 to
993d8d2
Compare
gs-olive
force-pushed
the
dynamo_compile_64bit_support
branch
from
7548ad2 to
9cf1785
Compare
gs-olive
force-pushed
the
fx_aten_64bit_support
branch
from
993d8d2 to
edf0bf7
Compare
gs-olive
force-pushed
the
dynamo_compile_64bit_support
branch
from
9cf1785 to
921dd2f
Compare
gs-olive
force-pushed
the
fx_aten_64bit_support
branch
from
edf0bf7 to
9a9820a
Compare
gs-olive
force-pushed
the
dynamo_compile_64bit_support
branch
from
921dd2f to
46b585b
Compare
gs-olive
force-pushed
the
fx_aten_64bit_support
branch
from
9a9820a to
183ba3a
Compare
gs-olive
force-pushed
the
dynamo_compile_64bit_support
branch
from
46b585b to
eca6d00
Compare
gs-olive
force-pushed
the
fx_aten_64bit_support
branch
from
183ba3a to
938dfad
Compare
gs-olive
force-pushed
the
dynamo_compile_64bit_support
branch
from
eca6d00 to
fb6f8eb
Compare
gs-olive
force-pushed
the
fx_aten_64bit_support
branch
from
938dfad to
dd3bad9
Compare
There was a problem hiding this comment.
There are some changes that do not conform to Python style guidelines:
--- py/torch_tensorrt/dynamo/fx_ts_compat/lower.py 2023-07-07 22:25:55.960522 +0000
+++ py/torch_tensorrt/dynamo/fx_ts_compat/lower.py 2023-07-07 22:26:11.127951 +0000
@@ -8,11 +8,15 @@
import torch.fx as fx
import torch.nn as nn
import torch_tensorrt.fx.tracer.dispatch_tracer.aten_tracer as aten_tracer
from torch.fx.passes.splitter_base import SplitResult
-from torch_tensorrt.dynamo.common import TRTInterpreter, TRTInterpreterResult, use_python_runtime_parser
+from torch_tensorrt.dynamo.common import (
+ TRTInterpreter,
+ TRTInterpreterResult,
+ use_python_runtime_parser,
+)
from .lower_setting import LowerSetting
from .passes.lower_pass_manager_builder import LowerPassManagerBuilder
from .passes.pass_utils import PassFunc, validate_inference
from torch_tensorrt.fx.tools.timing_cache_utils import TimingCacheManager
from torch_tensorrt.fx.tools.trt_splitter import TRTSplitter, TRTSplitterSetting
- Add `common` directory which stores code common to both the compile and export path, to reduce code duplication and better organize the repository - Update necessary imports, promote the `_defaults` module to `torch_tensorrt.dynamo._defaults`
- Add default, setting, and function arguments for `truncate_long_and_double` in Dynamo - Add utilities for repairing long/double inputs to TRT engines, including support for autocasting back to long/double after the computation completes - Add multiple helper functions to enable easy testing and diagnosis of long/double IO to TRT engines - Add necessary compiler code to enable usage of the `truncate_long_and_double` argument as a switch for the feature - Add Dynamo compile support for `truncate_long_and_double` compilation argument by intercepting long/double type inputs and casting them to their 32-bit counterparts prior to usage in TRT-accelerated subgraphs, then casting back if necessary - Add robust logic to handle 64-bit inputs and outputs - Add test cases for long and double scenarios - Centralize truncation utility for later use in Dynamo export path
gs-olive
force-pushed
the
dynamo_compile_64bit_support
branch
from
fb6f8eb to
21847e3
Compare
- Add `truncate_long_and_double` argument in FX settings to allow 64bit inputs - Utilize existing Dynamo functionality to repair FX aten graphs with 64bit inputs - Refactor imports in Dynamo to avoid circular import issue from new dependency issues - Add test cases to validate new feature
gs-olive
force-pushed
the
fx_aten_64bit_support
branch
from
dd3bad9 to
c4a0e9d
Compare
gs-olive
force-pushed
the
dynamo_compile_64bit_support
branch
2 times, most recently
from
dfa04ed to
9548c7f
Compare
|
Closing this in favor of #1983 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
truncate_long_and_doubleargument in FX settings to allow 64bit inputsNote: Breaks
python setup.py install --fx-onlyby introducing Dynamo dependency in FX. Potential resolutions include moving Dynamo utilities out of the Dynamo directory, or changing C++ requiring imports in Dynamo, etc.Fixes #2008
Type of change
Checklist: