Extend optimize_for_ort to cover passes #2274
Conversation
titaiwangms
requested review from
gramalingam,
shubhambhokare1,
justinchuby and
xadupre
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2274 +/- ##
==========================================
- Coverage 73.78% 73.77% -0.01%
==========================================
Files 235 235
Lines 30936 30939 +3
Branches 3494 3494
==========================================
Hits 22825 22825
- Misses 6911 6914 +3
Partials 1200 1200 ☔ View full report in Codecov by Sentry. |
|
Please also consider whether this method should be optimize in-place or not. I think we can make it in-place now that shape-inference itself is in-place. |
| # Apply the ORT pattern rewrite rules. | ||
| rewrite(model, ORT_PATTERN_REWRITE_RULES) | ||
|
|
||
| # TODO(exporter team): Fold transpose into initializers | ||
| # Apply the ORT optimization passes. | ||
| # https://github.com/microsoft/onnxruntime/blob/74dcf7e296639095dfa55d31336998b6f719ed76/onnxruntime/python/tools/transformers/dynamo_onnx_helper.py#L172 | ||
| common_passes.ClearMetadataAndDocStringPass()(model) |
There was a problem hiding this comment.
You may put all the passes into a pass manager like we do in optimize()
I think making it out-of-place is safer, in case we have passes in the future that need to be functional? |
| # https://github.com/microsoft/onnxruntime/blob/74dcf7e296639095dfa55d31336998b6f719ed76/onnxruntime/python/tools/transformers/dynamo_onnx_helper.py#L172 | ||
| common_passes.ClearMetadataAndDocStringPass(), | ||
| # https://github.com/microsoft/onnxruntime/blob/74dcf7e296639095dfa55d31336998b6f719ed76/onnxruntime/python/tools/transformers/dynamo_onnx_helper.py#L139 | ||
| common_passes.LiftConstantsToInitializersPass(lift_all_constants=False, size_limit=1), |
There was a problem hiding this comment.
We have another pass called LiftSubgraphInitializersToMainGraphPass. Do we know if it's needed in genAI? @kunal-vaishnavi
There was a problem hiding this comment.
If the pass logic is in DynamoOnnxHelper, then it is used for ONNX Runtime GenAI.
There was a problem hiding this comment.
We don't really produce graphs with subgraph initializers. I think we are ok either way
Fix #2261
A draft for discussion. We should cover all post-processing the model shipping needs