Closed
Description
Bug Description
When converting the deepset/roberta-base-squad2 Hugging Face network (https://huggingface.co/deepset/roberta-base-squad2) from TorchScript to Torch-TRT, the following error is encountered:
RuntimeError: [Error thrown at core/conversion/conversion.cpp:428] Expected eval_list->elements().size() == n->outputs().size() to be true but got false
Size of evaluated results: 3 and node outputs size: 2 must match.
To Reproduce
Steps to reproduce the behavior:
- Run
torch_tensorrt.compile
withdeepset/roberta-base-squad2
model as input, usingfp32
precision. - Choose fixed input size of
[1, 128]
and enabletruncate_long_and_double
with 8 GB workspace. - Pass in model keyword args to disable attention and hidden state outputs
Expected behavior
Model should successfully compile to Torch-TRT. Specifically, the size of evaluated results and node outputs should be matching
Environment
- Torch-TensorRT Version: 1.3.0a0+b70c9132
- PyTorch Version: 1.13.0.dev20220921+cu116
- CPU Architecture: Intel Xeon CPU
- OS: Ubuntu 20.04
- How you installed PyTorch: pip
- Build command you used: python setup.py develop
- Are you using local sources or building from archives: local
- Python version: 3.8.13
- CUDA version: 11.6
Additional context
Bug was reproduced on PyTorch 22.08 and 22.09 containers