🐛 [Bug] Crash or throw error with multiple inputs

[seed93]

Bug Description

Crash or throw error with multiple inputs

To Reproduce

import torch
import torch.nn as nn
from typing import Tuple, Dict, List
from torch import Tensor

class Model1(nn.Module):
    def __init__(self):
        super(Model1, self).__init__()
    def forward(self, inputs: List[Tuple[Tensor, Tensor]]):
        return inputs[0]

class ExportModel(nn.Module):
    def __init__(self, model):
        super(ExportModel, self).__init__()
        self.model = model
    def forward(self, t1, t2):
        outputs = self.model([(t1,t2)])
        return outputs[1]

export_model = ExportModel(Model1())
torch_script_module = torch.jit.script(export_model) 

input_list = [torch.tensor([[1,2,3]]), torch.tensor([[1,2,3]])]

result = torch_script_module(*input_list)

input_shape = []
for input in input_list:
    input_shape.append(input.shape)

compile_settings = {
    "input_shapes": input_shape,
    "op_precision": torch.float32,
    "torch_fallback" : {
      "enabled" : True,
      "min_block_size" : 1
    }
}
print(compile_settings)
trt_ts_module = trtorch.compile(torch_script_module, compile_settings)
result = trt_ts_module(*input_list)
torch.jit.save(trt_ts_module, "trt_torchscript_module.ts")

The code above will throw error as below:

~/.local/lib/python3.6/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

RuntimeError: forward() expected at most 2 argument(s) but received 3 argument(s). Declaration: forward.forward(Tensor t1.1, Tensor t2.1) -> (Tensor t2.1)

If I chahge the torch_fallback->enbled to False, it will get segmentation fault in trtorch.compile.

Expected behavior

convert a good model.

Environment

release v0.3.0

[narendasan]

Is this a duplicate of #474?

[seed93]

Is this a duplicate of #474?

No. This has different behavior and maybe new bugs.

[narendasan]

Even though your original graph is:

graph(%self : __torch__.ExportModel,
      %t1.1 : Tensor,
      %t2.1 : Tensor):
  %10 : int = prim::Constant[value=1]() # error1.py:19:23
  %3 : __torch__.Model1 = prim::GetAttr[name="model"](%self)
  %6 : (Tensor, Tensor) = prim::TupleConstruct(%t1.1, %t2.1)
  %7 : (Tensor, Tensor)[] = prim::ListConstruct(%6)
  %outputs.1 : (Tensor, Tensor) = prim::CallMethod[name="forward"](%3, %7) # error1.py:18:18
  %11 : Tensor = prim::TupleIndex(%outputs.1, %10)
  return (%11)

It reduces to

graph(%t1.1 : Tensor,
      %t2.1 : Tensor):
  return (%t2.1)

I do notice that the module isnt added as an argument to the forward. This is probably because there are no TRT components, its just returning you the lowered graph. With partial compilation off, you get an error from TRTorch saying that essentially there is nothing to be done, which is correct since you are just returning the input with no operations being done on it. I think the segfault is coming from just not catching the error, we can fix that

Seems like the program you provided is some what contrived, I'm not sure what you are attempting to do here.