Conversion Problem with SSD MobileNetv2 MNAS - Depthwise Convolutions

[phager90]

Describe the bug
Some depthwise convolutions are not converted properly in SSD MobileNetv2 MNAS.

Urgency
none

To Reproduce
Describe steps/code to reproduce the behavior:
Download SSD MobileNetv2 MNASNET: http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_mnasfpn_shared_box_predictor_320x320_coco_sync_2020_05_18.tar.gz
Convert with:

python -m tf2onnx.convert --graphdef ./frozen_inference_graph.pb --output out.onnx \
        --fold_const --opset 10 \
        --inputs normalized_input_image_tensor:0 \
        --outputs raw_outputs/class_predictions:0,raw_outputs/box_encodings:0

Expected behavior
Depthwise convolutions on heads (frozen graph file):

Group properly not correctly set on out.onnx:

Additional context
Output is quite erratic. On some depthwise convolutions is it correct, on others not. It could be related to the fact that the weights are broadcasted from a static node to multiple convolutions.

It looks that only one of the Depthwise Conv receiving the weights seems to be converted correctly:

[guschmue]

let me take a look.

[TomWildenhain-Microsoft]

@phager90 Are you still having this issue?

[phager90]

@TomWildenhain-Microsoft Yes I am still having this issue.

[TomWildenhain-Microsoft]

@phager90 you were 100% correct with your diagnosis of the issue. The depthwise conv handler was mutating the weights matrix, which caused an issue when multiple convs shared it.

[WangFengtu1996]

Hi, all

I got a error, execute the command ,input the same pre-train model.

command

python -m tf2onnx.convert --graphdef ./frozen_inference_graph.pb --output out.onnx \
        --fold_const --opset 10 \
        --inputs normalized_input_image_tensor:0 \
        --outputs raw_outputs/class_predictions:0,raw_outputs/box_encodings:0 

error

/home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/runpy.py:125: RuntimeWarning: 'tf2onnx.convert' found in sys.modules after import of package 'tf2onnx', but prior to execution of 'tf2onnx.convert'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
WARNING:tensorflow:From /home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/site-packages/tf2onnx/verbose_logging.py:76: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

2021-07-30 11:34:29,009 - WARNING - From /home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/site-packages/tf2onnx/verbose_logging.py:76: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

Traceback (most recent call last):
  File "/home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/site-packages/tensorflow_core/python/framework/importer.py", line 501, in _import_graph_def_internal
    graph._c_graph, serialized, options)  # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.InvalidArgumentError: NodeDef mentions attr 'exponential_avg_factor' not in Op<name=FusedBatchNormV3; signature=x:T, scale:U, offset:U, mean:U, variance:U -> y:T, batch_mean:U, batch_variance:U, reserve_space_1:U, reserve_space_2:U, reserve_space_3:U; attr=T:type,allowed=[DT_HALF, DT_BFLOAT16, DT_FLOAT]; attr=U:type,allowed=[DT_FLOAT]; attr=epsilon:float,default=0.0001; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]; attr=is_training:bool,default=true>; NodeDef: {{node FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNormV3}}. (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/site-packages/tf2onnx/convert.py", line 605, in <module>
    main()
  File "/home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/site-packages/tf2onnx/convert.py", line 213, in main
    graph_def, inputs, outputs = tf_loader.from_graphdef(args.graphdef, args.inputs, args.outputs)
  File "/home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/site-packages/tf2onnx/tf_loader.py", line 313, in from_graphdef
    tf.import_graph_def(graph_def, name='')
  File "/home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/site-packages/tensorflow_core/python/framework/importer.py", line 405, in import_graph_def
    producer_op_list=producer_op_list)
  File "/home/yons/workspace/anaconda3/envs/nx_cross_compilers/lib/python3.7/site-packages/tensorflow_core/python/framework/importer.py", line 505, in _import_graph_def_internal
    raise ValueError(str(e))
ValueError: NodeDef mentions attr 'exponential_avg_factor' not in Op<name=FusedBatchNormV3; signature=x:T, scale:U, offset:U, mean:U, variance:U -> y:T, batch_mean:U, batch_variance:U, reserve_space_1:U, reserve_space_2:U, reserve_space_3:U; attr=T:type,allowed=[DT_HALF, DT_BFLOAT16, DT_FLOAT]; attr=U:type,allowed=[DT_FLOAT]; attr=epsilon:float,default=0.0001; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]; attr=is_training:bool,default=true>; NodeDef: {{node FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNormV3}}. (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).

Env

• ubuntu 18.04
• python Python 3.7.10
• tensorflow-estimator 1.15.1
• tensorflow-gpu 1.15.5
• onnx 1.6.0
• onnxruntime 1.3.0
• tf2onnx 1.9.1

[WangFengtu1996]

HI, @phager90 @guschmue @TomWildenhain-Microsoft
I also try different version --opset 9, 10, 11 , but it also not work

[TomWildenhain-Microsoft]

@TUT-jiayou which model is this for? Are you sure your tensorflow is up do date with the version that generated the model? It seems like tensorflow is unable to open the model.

[WangFengtu1996]

@TomWildenhain-Microsoft
The model
(http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_mnasfpn_shared_box_predictor_320x320_coco_sync_2020_05_18.tar.gz) are converted.

About the TF version matching problem, I just know that it use tensorflow 1 version.

[TomWildenhain-Microsoft]

Since the graph is frozen, it can be opened with tf2. I used tensorflow==2.5.0 and the same command as you and it converted without issue. I think the frozen graph might have been generated with tf2 even if the original model used tf1.