Pythoncall: Support arrays of int, real as args

[Shaikh-Ubaid]

towards #703

This PR supports passing of arrays of types int, real to @ pythoncall decorated functions. Returning arrays of simple types is yet to be supported.

[Shaikh-Ubaid]

I updated the cmake minimum version here. cmake with version 3.5 is unable to find python from the conda environment (It always picks the system level python). It seems cmake with version 15 and above supports finding python from conda environment (reference https://gitlab.kitware.com/cmake/cmake/-/issues/21322#note_845923). Python from conda environment is needed as the conda environment contains the installed numpy package.

[Shaikh-Ubaid]

Doubt related to return types as arrays:

$ cat expr2.py
from lpython import pythoncall, f32

@pythoncall(module = "expr2_module")
def return_multi_dim_arr() -> f32[2, 3]:
    pass

def main0():
    x: f32[2, 3] = return_multi_dim_arr()
    print("x = ", x)

main0()
$ cat expr2_module.py 
import numpy as np

def return_multi_dim_arr():
    a = np.array([3.12, -2.14, 1., 4., 5., 6.]).reshape((2, 3))
    print("CPython: ", a)
    return a

In the above example, the function def return_multi_dim_arr() -> f32[2, 3] gets transformed to def return_multi_dim_arr(struct r32* _lpython_return_variable) (I think via the create_subroutine_from_function() and/or array_op pass). Since it is transformed, its signature does not match to that of its counter-part defined in the expr2_module.py.

How do we tackle this? Shall we skip updating/modifying @pythoncall decorated functions in the create_subroutine_from_function() and array_op pass?

CC: @certik, @czgdp1807, @Smit-create

[certik]

I would call it

Suggested change

	app.add_flag("--enable-cnumpy", compiler_options.enable_cnumpy, "Enable C-Numpy runtime");
	app.add_flag("--enable-numpy", compiler_options.enable_numpy, "Enable NumPy runtime (implies --enable-cpython)");

[certik]

Shall we skip updating/modifying @pythoncall decorated functions in the create_subroutine_from_function() and array_op pass?

Yes. I think it skips BindC functions and we just need it to also skip BindPython functions too.

[certik]

I think this looks great!

[certik]

I am going to merge this, as it looks good. In subsequent PRs:

• Use --link-numpy (see my comment above)
• Create a dedicated CI and update our integration_tests, that tests everything except python, via C and LLVM backends and ensures we have no dependency on CPython; and add a CI that only tests the Python interoperability tests (for now only C, later LLVM as well)

[Shaikh-Ubaid]

Use --enable-numpy

We can support this. A minor concern here is that people/users might get confused as numpy is also package that can be imported (In this PR, cnumpy refers to numpy-c library/API). For example, users might think --enable-numpy is used to enable importing numpy, that is, from numpy import empty, .... In actual --enable-numpy is only needed if you are using numpy along with support for cpython runtime.

[certik]

I see. Maybe we can call it --link-numpy instead then.