ORT nightly version 1.11.0.dev20220311003 supports numpy <= 1.21

[hwangdeyu]

Describe the bug:
Tried to validate ORT nightly version 1.11.0.dev20220311003, but it only supports numpy version >= 1.21.0 when we run the install command.
However, many of the tf2onnx ci tests were failed, cause it cannot convert a symbolic Tensor (x:0) to a numpy array with numpy version 1.21.x.
stackoverflow: Cannot convert a symbolic Tensor (2nd_target:0) to a numpy array
TF github issue: Cannot convert a symbolic Tensor (strided_slice:0) to a numpy array.
So I want to know if is it possible to support numpy == 1.19.x in ORT version 1.11.0?

System information:
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 20.04
ONNX Runtime installed from (source or binary): pip install --index-url https://test.pypi.org/simple/ ort-nightly==1.11.0.dev20220311003
ONNX Runtime version: ort-nightly 1.11.0.dev20220311003
Python version: Python 3.7
Tensorflow version: 2.x

Expected behavior
It can support numpy == 1.19.x.

[snnn]

The error says:"This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported"

tf.Tensor has a "numpy()" method, it seems you forgot to use it?

[hwangdeyu]

Actually, all the tf2onnx ci tests is passed when numpy==1.19.x. And it would be failed in tensorflow source once we use numpy==1.21.x for ort-nightly 1.11 .
There are some examples of reproducing the error in tf repo issue. tensorflow/tensorflow#47957.
I am not sure if it's possible for latest ort 1.11 to support numpy==1.19.x, which can be compatible to support more versions of frameworks such as TensorFlow.

[garymm]

@hwangdeyu can you force numpy to 1.19 in CI?

Which versions of TensorFlow don't support numpy 1.21? At some point they added support. Can we just configure the CI pipelines for newer TF versions to use ORT 1.11?

[hwangdeyu]

@hwangdeyu Deyu Huang FTE can you force numpy to 1.19 in CI?

Which versions of TensorFlow don't support numpy 1.21? At some point they added support. Can we just configure the CI pipelines for newer TF versions to use ORT 1.11?

Sure, I will try to test it with numpy 1.19 after ort install successfully.

[fatcat-z]

@snnn

May I know why the latest version of ORT could not accept a lower version of numpy, like 1.19? Is there any new features relying on the new version of numpy?

[snnn]

It just because the package was built with numpy 1.21, so at runtime it requires numpy 1.21. No other reasons. You can make your own build with lower numpy versions.

[garymm]

@snnn can you change the package requirements to be more lenient? It seems at runtime it actually succeeds if we force install numpy 1.19.

[faxu]

I believe currently the automated pipelines only validate the python package we build (using 1.21), and there is no automated test coverage for older versions. Relaxing the numpy version would either:
a) require adding more automated test pipelines for older version -> not sustainable
b) not add the automated tests, and risk older versions not working -> user issues

Out of these, it seems like requiring the tested version may be the best option. In general it's good practice to update to newer packages and for numpy, we specifically upgraded to 1.21 due to a security vulnerability in the older version.

For tf2onnx:

• for older version of TF that don't support numpy 1.21 -> use older version of ORT, or manually ORT with numpy 1.19
• for newer versions of TF that do support 1.21 -> directly use ORT 1.11

[garymm]

Seems reasonable.