You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Certain models, including T5 (see #1864, #1740) require INT64 inputs to their embedding layers. 64-bit input types are not compatible with TRT Engines, however, thus a truncation mechanism is necessary to resolve this issue.
Feature Proposal
Add support for truncate_long_and_double in Dynamo compile, tracking data types and automatically casting to TRT-allowed types (and back) as necessary.