Skip to content

Fix ONNXRT calibration for Dml EP #1526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Jan 18, 2024
Merged

Fix ONNXRT calibration for Dml EP #1526

merged 20 commits into from
Jan 18, 2024

Conversation

yuwenzho
Copy link
Contributor

@yuwenzho yuwenzho commented Jan 10, 2024

Type of Change

bug fix
API changed or not: no

Description

Fix ONNXRT calibration for Dml EP: calibrate initializer tensors and output tensors separately.

The bug is found in microsoft/Olive#852

How has this PR been tested?

CI

Dependency Change?

no

@yuwenzho yuwenzho added the bug fix Something isn't working label Jan 10, 2024
@chensuyue chensuyue added this to the v2.5 milestone Jan 12, 2024
@yuwenzho yuwenzho requested a review from mengniwang95 January 16, 2024 11:53
@yuwenzho
Copy link
Contributor Author

extension test on Dml EP of ssd_mobilenet_v1, shufflenet-v2-12.

Acc regression of resnet50-v1-12 in CI model test is reasonable: The graph optimization level here is changed from 'ORT_ENABLE_ALL' (default) to 'ORT_DISABLE_ALL', which will casuse the intermediate output to change slightly.

@chensuyue
Copy link
Contributor

extension test on Dml EP of ssd_mobilenet_v1, shufflenet-v2-12.

Acc regression of resnet50-v1-12 in CI model test is reasonable: The graph optimization level here is changed from 'ORT_ENABLE_ALL' (default) to 'ORT_DISABLE_ALL', which will casuse the intermediate output to change slightly.

Ok, then we need to update model refer before merge this PR. But why need to disable the graph optimization?

@yuwenzho
Copy link
Contributor Author

extension test on Dml EP of ssd_mobilenet_v1, shufflenet-v2-12.
Acc regression of resnet50-v1-12 in CI model test is reasonable: The graph optimization level here is changed from 'ORT_ENABLE_ALL' (default) to 'ORT_DISABLE_ALL', which will casuse the intermediate output to change slightly.

Ok, then we need to update model refer before merge this PR. But why need to disable the graph optimization?

The graph optimization is disabled during calibration, becasue:

  1. The model has been graph optimized before, so there is no need to optimize further during the calibration phase.
  2. If the EP is a Dml EP and further graph optimization is performed, the model intermediate outputs required for calibration cannot be collected correctly.
  3. Refer to the session creation func in onnxruntime calibration.

@chensuyue chensuyue merged commit c58aeaa into master Jan 18, 2024
@chensuyue chensuyue deleted the yuwenzho/fix_calib branch January 18, 2024 06:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug fix Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants