Support shape analysis for dynamic fallback

[Njuapp]

Signed-off-by: Cheng Hang [email protected]

Description

Previously, when encountered ops/modules that have to fall back and meanwhile dynamic shape is used, shape analysis would fail when doing generateRandomInputs(), because tensors with -1 contained in shape cannot be created.

To support dynamic fallback, we have to firstly overcome this failure, by doing generateRandomInputs() with min/max/opt shape info. Then shape analysis can be executed three times, with min/max/opt shapes correspondingly.

Fixes #1113

Type of change

Please delete options that are not relevant and/or add your own.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified

[narendasan]

I think this feature needs a formal design. Please open an RFC for this. For example just through discussion about this feature one issue we thought of is that TensorRT has limitations in number of free dimensions in dynamic shape for some layers. We should augment the converter library to report these cases back so that we can check for support before moving forward. I suspect there are other similar cases