converter only supports while loops with a single scan output

[wangqiaoshi]

Describe the bug
A clear and concise description of what the bug is.

• convert a transformer model
• training code https://github.com/OpenNMT/OpenNMT-tf
• throw exception

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/tf2onnx/tfonnx.py", line 289, in tensorflow_onnx_mapping
    func(g, node, **kwargs, initialized_tables=initialized_tables)
  File "/opt/conda/lib/python3.8/site-packages/tf2onnx/onnx_opset/controlflow.py", line 444, in version_7
    utils.make_sure(len(removed_scan_outputs) <= 1, "converter only supports while loops with a single scan output")
  File "/opt/conda/lib/python3.8/site-packages/tf2onnx/utils.py", line 218, in make_sure
    raise ValueError("make_sure failure: " + error_msg % args)
ValueError: make_sure failure: converter only supports while loops with a single scan output

Urgency
If there are particular important use cases blocked by this or strict project-related timelines, please share more information and dates. If there are no hard deadlines, please specify none.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): 18.04
• Tensorflow Version: 2.3.1
• Python version: 3.8

To Reproduce
Describe steps/code to reproduce the behavior:

wget https://s3.amazonaws.com/opennmt-models/averaged-ende-export500k-v2.tar.gz
tar xf averaged-ende-export500k-v2.tar.gz
python -m tf2onnx.convert --saved-model averaged-ende-export500k-v2 --extra_opset ai.onnx.contrib:1 --opset 10 --output model.onnx

Expected behavior

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here. If the issue is about a particular model, please share the model details as well to facilitate debugging.

[wangqiaoshi]

@guschmue Excuse me, sorry, is there a solution for this?
I have a question, why do you say that there is no equivalent step in onnx and we should remove it.Onnx should support TensorListReserve, TensorListResize.
https://github.com/onnx/tensorflow-onnx/blob/master/support_status.md

[guschmue]

onnx does not support TensorListReserve and TensorListResize - basically in tensorflow the body of a loop will maintain a list, grow it and append things to it. In onnx the loop operator is maintaining the list and the body just yields the elements.
We have not seem much problems mapping the tf loop to the onnx loop this way.
In opset-11 onnx introduced sequences and they are somewhat similar to tf lists but need have not seen the need to use them.

Not sure about your specific issue - we need to look at it.

[TomWildenhain-Microsoft]

Yeah, I'll take a look at this.

[wangqiaoshi]

onnx does not support TensorListReserve and TensorListResize - basically in tensorflow the body of a loop will maintain a list, grow it and append things to it. In onnx the loop operator is maintaining the list and the body just yields the elements.
We have not seem much problems mapping the tf loop to the onnx loop this way.
In opset-11 onnx introduced sequences and they are somewhat similar to tf lists but need have not seen the need to use them.

Not sure about your specific issue - we need to look at it.

@guschmue If write like this,converter will be a problem

import tensorflow as tf

class MyModel(tf.keras.layers.Layer):
    def __init__(self):
        super(MyModel, self).__init__()
        self.index = tf.constant(0)



    @tf.function(
        input_signature=[tf.TensorSpec(shape=(5, 5), dtype=tf.float32)])
    def call(self, inputs):
        output_ta = tf.TensorArray(dtype=tf.int32, size=0, dynamic_size=True)
        output_ta_v2 = tf.TensorArray(dtype=tf.int32, size=0, dynamic_size=True)
        cond = lambda i, *_: tf.logical_and(tf.less(i, 10), i >= 0)
        def body(i, out_ta,out_ta_v2):
            new_i = tf.add(i, 1)
            out_ta_new = out_ta.write(i, i)
            out_ta_new_v2 = out_ta_v2.write(i,i+1)
            return new_i, out_ta_new,out_ta_new_v2
        i_final, ta_final,ta_final_v2 = tf.while_loop(cond, body, [self.index, output_ta,output_ta_v2])
        output = tf.add(ta_final.stack(),ta_final_v2.stack())
        return output

Do you mean that writing like this is not standardized,'basically in tensorflow the body of a loop will maintain a list' means that tensorflow will generate a list by default?

This usage is quite common, such as in the decoder of the transformer. eg
https://github.com/OpenNMT/OpenNMT-tf/blob/master/opennmt/utils/decoding.py#L377-L510

[guschmue]

onnx does not have something like a tensor array (maybe the new sequence ops in opset-11 are close). But we can have the loop operator assemble the list by yielding the elements. Only condition is that it needs to be a sequential write to the tensor array. I think we should be able to make the example above work.