🐛 [Bug] EfficientDet conversion failed

[vinhngx]

Bug Description

EfficientDet conversion failed with:

RuntimeError: [Error thrown at core/conversion/conversion.cpp:118] Expected converter to be true but got false
Unable to convert node: %25577 : int = aten::Int(%25576), scope: __module.backbone/__module.backbone.conv_stem (conversion.AddLayer)
Schema: aten::Int.Tensor(Tensor a) -> (int)
Converter for aten::Int requested, but no such converter was found.

To Reproduce

A prototype notebook is available at:
https://github.com/vinhngx/TRTorch/blob/efficientdet/notebooks/efficientdet-object-detection-demo.ipynb

Steps to reproduce the behavior:

1. Docker image: ./docker/Dockerfile.21.06

Expected behavior

Environment

Build information about the TRTorch compiler can be found by turning on debug messages

• TRTorch Version (e.g. 0.2.0):
• PyTorch Version (e.g. 1.0):
• CPU Architecture:
• OS (e.g., Linux):
• How you installed PyTorch (conda, pip, libtorch, source):
• Build command you used (if compiling from source):
• Are you using local sources or building from archives:
• Python version:
• CUDA version:
• GPU models and configuration:
• Any other relevant information:

Additional context

[narendasan]

We will need to patch this to get it to work. Since you are pulling from a github repo, we can just fork it and apply the patch

[narendasan]

Also another option is to get scripting for the model working, I believe this will solve this problem

[ncomly-nvidia]

@vinhngx have you been able to attempt scripting yet?

[ncomly-nvidia]

Also, @vinhngx, have you been able to test with fallback on?

[vinhngx]

@vinhngx have you been able to attempt scripting yet?

scritping path together with fallback

trt_scripted_model = trtorch.compile(script_model, {
    "inputs": [trtorch.Input((3, 3, 512, 512))],
    "enabled_precisions": {torch.float, torch.half}, # Run with FP16
    "workspace_size": 1 << 20,
    "truncate_long_and_double": True,
    "torch_fallback": {
        "enabled": True
    }
})

failed with the following

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_465/1502127812.py in <module>
----> 1 trt_scripted_model = trtorch.compile(script_model, {
      2     "inputs": [trtorch.Input((3, 3, 512, 512))],
      3     "enabled_precisions": {torch.float, torch.half}, # Run with FP16
      4     "workspace_size": 1 << 20,
      5     "truncate_long_and_double": True,

/opt/conda/lib/python3.8/site-packages/trtorch/_compiler.py in compile(module, compile_spec)
     79             "torch.jit.ScriptFunction currently is not directly supported, wrap the function in a module to compile")
     80 
---> 81     compiled_cpp_mod = trtorch._C.compile_graph(module._c, _parse_compile_spec(compile_spec))
     82     compiled_module = torch.jit._recursive.wrap_cpp_module(compiled_cpp_mod)
     83     return compiled_module

RuntimeError: 
Couldn't export Python method.:
  File "/opt/conda/lib/python3.8/site-packages/effdet/efficientdet.py", line 454
            return self._forward_level_first(x)
        else:
            return self._forward(x)
                   ~~~~~~~~~~~~~ <--- HERE

[vinhngx]

Also, @vinhngx, have you been able to test with fallback on?

tracing path together with fallback:

import trtorch

# The compiled module will have precision as specified by "op_precision".
# Here, it will have FP16 precision.
trt_model = trtorch.compile(traced_model, {
    "inputs": [trtorch.Input((3, 3, 512, 512))],
    "enabled_precisions": {torch.float, torch.half}, # Run with FP16
    "workspace_size": 1 << 20, 
    "truncate_long_and_double": True,
    "torch_fallback": {
        "enabled": True
    }
})

Failed with:

ERROR: [TRTorch] - 4: [layers.cpp::validate::2291] Error Code 4: Internal Error (%0 : Tensor = aten::floor_divide(%pad_h.1, %2), scope: __module.backbone/__module.backbone.conv_stem # /opt/conda/lib/python3.8/site-packages/torch/_tensor.py:557:0: operation FLOOR_DIV has incompatible input types Float and Int32)
ERROR: [TRTorch] - 4: [layers.cpp::validate::2291] Error Code 4: Internal Error (%0 : Tensor = aten::floor_divide(%pad_h.1, %2), scope: __module.backbone/__module.backbone.conv_stem # /opt/conda/lib/python3.8/site-packages/torch/_tensor.py:557:0: operation FLOOR_DIV has incompatible input types Float and Int32)
ERROR: [TRTorch] - 4: [layers.cpp::validate::2291] Error Code 4: Internal Error (%0 : Tensor = aten::floor_divide(%pad_h.1, %2), scope: __module.backbone/__module.backbone.conv_stem # /opt/conda/lib/python3.8/site-packages/torch/_tensor.py:557:0: operation FLOOR_DIV has incompatible input types Float and Int32)
ERROR: [TRTorch] - 2: [builder.cpp::buildSerializedNetwork::417] Error Code 2: Internal Error (Assertion enginePtr != nullptr failed.)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_465/244551165.py in <module>
      3 # The compiled module will have precision as specified by "op_precision".
      4 # Here, it will have FP16 precision.
----> 5 trt_model = trtorch.compile(traced_model, {
      6     "inputs": [trtorch.Input((3, 3, 512, 512))],
      7     "enabled_precisions": {torch.float, torch.half}, # Run with FP16

/opt/conda/lib/python3.8/site-packages/trtorch/_compiler.py in compile(module, compile_spec)
     79             "torch.jit.ScriptFunction currently is not directly supported, wrap the function in a module to compile")
     80 
---> 81     compiled_cpp_mod = trtorch._C.compile_graph(module._c, _parse_compile_spec(compile_spec))
     82     compiled_module = torch.jit._recursive.wrap_cpp_module(compiled_cpp_mod)
     83     return compiled_module

RuntimeError: [Error thrown at core/conversion/conversionctx/ConversionCtx.cpp:160] Building serialized network failed in TensorRT

[github-actions]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

[github-actions]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days