Skip to content

🐛 [Bug] Encountered bug when converting deepset/roberta-base-squad2 Hugging Face Network #1410

Closed
@gs-olive

Description

@gs-olive

Bug Description

When converting the deepset/roberta-base-squad2 Hugging Face network (https://huggingface.co/deepset/roberta-base-squad2) from TorchScript to Torch-TRT, the following error is encountered:

RuntimeError: [Error thrown at core/conversion/conversion.cpp:428] Expected eval_list->elements().size() == n->outputs().size() to be true but got false
Size of evaluated results: 3 and node outputs size: 2 must match.

To Reproduce

Steps to reproduce the behavior:

  1. Run torch_tensorrt.compile with deepset/roberta-base-squad2 model as input, using fp32 precision.
  2. Choose fixed input size of [1, 128] and enable truncate_long_and_double with 8 GB workspace.
  3. Pass in model keyword args to disable attention and hidden state outputs

Expected behavior

Model should successfully compile to Torch-TRT. Specifically, the size of evaluated results and node outputs should be matching

Environment

  • Torch-TensorRT Version: 1.3.0a0+b70c9132
  • PyTorch Version: 1.13.0.dev20220921+cu116
  • CPU Architecture: Intel Xeon CPU
  • OS: Ubuntu 20.04
  • How you installed PyTorch: pip
  • Build command you used: python setup.py develop
  • Are you using local sources or building from archives: local
  • Python version: 3.8.13
  • CUDA version: 11.6

Additional context

Bug was reproduced on PyTorch 22.08 and 22.09 containers

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions