Onnx can not load the onnx model generated by tf2onnx

[wangzhenyu32]

code
import tensorflow as tf
from tf2onnx import tfonnx
import os
import onnx
import onnx.utils

def _save_onnx_model(model_file):
with tf.compat.v1.Session(graph=tf.Graph()) as session, open(model_file, mode='wb') as graph_file:
input_x = tf.compat.v1.placeholder(dtype=tf.float32, shape=[None, 2, 3, 4], name='x')
input_y = tf.compat.v1.placeholder(dtype=tf.float32, shape=[None, 2, 3, 4], name='y')
weight = tf.Variable(initial_value=4.2, dtype=tf.float32)
tf.multiply(input_x + input_y, weight, name='z')
session.run(weight.initializer)
onnx_model = tfonnx.process_tf_graph(tf_graph=session.graph).make_model(graph_doc='Test onnx model')
graph_file.write(onnx_model.SerializeToString())

model = onnx.load(model_file)
model = onnx.utils.polish_model(model)

if name == 'main':
SCRIPT_APTH = os.path.dirname(file)
model_file_name = os.path.join(SCRIPT_APTH, 'ci2.onnx')
_save_onnx_model(model_file_name)

error_messge:
2020-10-16 09:29:20.158155: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-10-16 09:29:20.158172: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)
2020-10-16 09:29:20.158184: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (wangzhenyucom): /proc/driver/nvidia/version does not exist
2020-10-16 09:29:20.158317: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-10-16 09:29:20.180417: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3000000000 Hz
2020-10-16 09:29:20.180719: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f219c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-10-16 09:29:20.180734: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
WARNING:tensorflow:From /home/wangzhenyu/code/env38/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py:1659: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
Tensorflow op [Variable: VarHandleOp] is not supported
Tensorflow op [z/ReadVariableOp: ReadVariableOp] is not supported
Tensorflow op [Variable/Read/ReadVariableOp: ReadVariableOp] is not supported
Tensorflow op [Variable/IsInitialized/VarIsInitializedOp: VarIsInitializedOp] is not supported
Tensorflow op [Variable/Assign: AssignVariableOp] is not supported
Unsupported ops: Counter({'ReadVariableOp': 2, 'VarHandleOp': 1, 'VarIsInitializedOp': 1, 'AssignVariableOp': 1})
Traceback (most recent call last):
File "/home/wangzhenyu/Desktop/data/testonnx/test_tf_onnx.py", line 50, in
_save_onnx_model(model_file_name)
File "/home/wangzhenyu/Desktop/data/testonnx/test_tf_onnx.py", line 44, in _save_onnx_model
model = onnx.utils.polish_model(model)
File "/home/wangzhenyu/code/env38/lib/python3.8/site-packages/onnx/utils.py", line 18, in polish_model
onnx.checker.check_model(model)
File "/home/wangzhenyu/code/env38/lib/python3.8/site-packages/onnx/checker.py", line 93, in check_model
C.check_model(model.SerializeToString())
onnx.onnx_cpp2py_export.checker.ValidationError: No Op registered for VarHandleOp with domain_version of 8

==> Context: Bad node spec: output: "Variable:0" name: "Variable" op_type: "VarHandleOp" attribute { name: "container" s: "" type: STRING } attribute { name: "dtype" i: 1 type: INT } attribute { name: "shape" type: FLOATS } attribute { name: "shared_name" s: "Variable" type: STRING }

Process finished with exit code 1

package config:
onnx 1.7.0
tf2onnx 1.7.1
tensorflow 2.2.0

[TomWildenhain-Microsoft]

You must freeze the graph before feeding it to the converter. This will remove the variable ops and make them constant.

import tensorflow as tf
from tf2onnx import tfonnx
import os
import onnx
import onnx.utils

def _save_onnx_model(model_file):
    with tf.compat.v1.Session(graph=tf.Graph()) as session:
        input_x = tf.compat.v1.placeholder(dtype=tf.float32, shape=[None, 2, 3, 4], name='x')
        input_y = tf.compat.v1.placeholder(dtype=tf.float32, shape=[None, 2, 3, 4], name='y')
        weight = tf.Variable(initial_value=4.2, dtype=tf.float32)
        tf.multiply(input_x + input_y, weight, name='z')
        session.run(weight.initializer)
        from tf2onnx import tf_loader
        frozen_graph_def = tf_loader.freeze_session(session, input_names=['x:0', 'y:0'], output_names=['z:0'])
    with tf.compat.v1.Session(graph=tf.Graph()) as session, open(model_file, mode='wb') as graph_file:
        tf.import_graph_def(frozen_graph_def, name='')
        onnx_model = tfonnx.process_tf_graph(tf_graph=session.graph).make_model(graph_doc='Test onnx model')
        graph_file.write(onnx_model.SerializeToString())

_save_onnx_model("test.onnx")

[TomWildenhain-Microsoft]

Let me know if this doesn't work or we can close the issue.

[wangzhenyu32]

Let me know if this doesn't work or we can close the issue.

It'd still doesn't work when I use:

model = onnx.load(model_file_name)
model = onnx.utils.polish_model(model)

error message:
2020-10-21 01:30:01.270012: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-10-21 01:30:01.270027: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)
2020-10-21 01:30:01.270041: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (wangzhenyucom): /proc/driver/nvidia/version does not exist
2020-10-21 01:30:01.270193: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-10-21 01:30:01.292340: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3000000000 Hz
2020-10-21 01:30:01.292644: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f2a8c000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-10-21 01:30:01.292670: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
WARNING:tensorflow:From /home/wangzhenyu/code/env38/lib/python3.8/site-packages/tensorflow/python/ops/resource_variable_ops.py:1659: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /home/wangzhenyu/code/env38/lib/python3.8/site-packages/tf2onnx/tf_loader.py:146: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.convert_variables_to_constants
WARNING:tensorflow:From /home/wangzhenyu/code/env38/lib/python3.8/site-packages/tensorflow/python/framework/graph_util_impl.py:359: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.extract_sub_graph
Traceback (most recent call last):
File "/home/wangzhenyu/Desktop/data/testonnx/test_tf_onnx.py", line 28, in
model = onnx.utils.polish_model(model)
File "/home/wangzhenyu/code/env38/lib/python3.8/site-packages/onnx/utils.py", line 18, in polish_model
onnx.checker.check_model(model)
File "/home/wangzhenyu/code/env38/lib/python3.8/site-packages/onnx/checker.py", line 93, in check_model
C.check_model(model.SerializeToString())
onnx.onnx_cpp2py_export.checker.ValidationError: Graph must be in single static assignment (SSA) form, however 'z/ReadVariableOp__3' has been used as graph input names multiple times.

Please pay attention to the tf2onnx version is the latest version.

[TomWildenhain-Microsoft]

Inference seems to be working for me but I agree that it is strange there appears to be a duplicate initializer. Looking into it now.

[TomWildenhain-Microsoft]

The duplicate initializer was caused by an aliasing bug. Fixing it in #1143