Initial support for @pythoncall decorator

[Shaikh-Ubaid]

towards #703.

[Shaikh-Ubaid]

Example:

$ cat examples/expr3.py 
def main0():
    n: i32 = 5
    m: i32 = fun_from_py_module(n)
    print("hi")
    print("m recieved is: ", m)
    print("this time value received is:", fun_from_py_module(-22))
    print("And this time value received is:", fun_from_py_module(101))

    print("cpython_average(2, 3): ", cpython_average(2, 3))
    print("cpython_average(-100, -1): ", cpython_average(-100, -1))

    cpython_print_str("hi how are you?")
    pqr: str = "abcdefgh"
    cpython_print_str("hi how are you?: " + pqr)

    abc: str = cpython_return_random_str()
    print("abc: ", abc)

    print(cpython_str_concat(abc, pqr))

@pythoncall(module="py_module")
def cpython_str_concat(a: str, b: str)->str:
    pass

@pythoncall(module="py_module")
def fun_from_py_module(n: i32) -> i32:
    pass

@pythoncall(module="py_module")
def cpython_average(n: i32, m: i32) -> f64:
    pass

@pythoncall(module="py_module")
def cpython_print_str(x: str):
    pass

@pythoncall(module="py_module")
def cpython_return_random_str() -> str:
    pass

main0()
$ cat py_module.py 
def fun_from_py_module(n):
    import os
    print(os.getcwd())
    print("hi from cpython", n)
    return n + 2

def cpython_average(n, m):
    return (m + n) / 3

def cpython_print_str(x: str):
    print("I received the string:", x)

def cpython_return_random_str() -> str:
    return "this is some string"

def cpython_str_concat(a: str, b: str)->str:
    return "concatenated: " + a + " " + b
$ lpython examples/expr3.py --backend c
/Users/ubaid/Desktop/OpenSource/lpython
hi from cpython 5
hi
m recieved is:  7
/Users/ubaid/Desktop/OpenSource/lpython
hi from cpython -22
this time value received is: -20
/Users/ubaid/Desktop/OpenSource/lpython
hi from cpython 101
And this time value received is: 103
cpython_average(2, 3):  1.666667
cpython_average(-100, -1):  -33.666667
I received the string: hi how are you?
I received the string: hi how are you?: abcdefgh
abc:  this is some string
concatenated: this is some string abcdefgh

Note: py_module.py is in the root directory of the lpython project.

[Shaikh-Ubaid]

Work In Progress (WIP): CI Tests may/might not pass.

[Shaikh-Ubaid]

Example from #703 (comment),

$ cat examples/expr2.py 

@pythoncall(module="py_module")
def f(n: i32) -> str:
    pass

def main0():
    p: str
    p = f(3)
    print(p)

main0()

$ cat py_module.py 
def f(n) -> str:
    import sympy
    sympy.var("x")
    e = ((x+5)**n).expand()
    return str(e)
$ lpython examples/expr2.py --backend c         
x**3 + 15*x**2 + 75*x + 125

[Shaikh-Ubaid]

I think this should be optional. We need these flags when there is a @pythoncall() in the user source code. Shall we add a compiler flag --enable-cpython-runtime? We also need the python version here so that we can link to the pythonlib. How do we get the python version that is installed on the system?

[certik]

Maybe just --enable-cpython and the Python version can be either specified at the command line. Alternatively we can specify the include path and the library name and path directly.

[Shaikh-Ubaid]

I added the flag --enable-cpython. Currently, I hardcoded the python_version and the include paths for python library.

[Shaikh-Ubaid]

How do we add support for import path (-I option)? The import paths are available with lpython, but not available with the generated C code. It is the generated C code which tries to open/read the module_file and evaluate the function present in it. I think we would need to provide/embed all the import paths into the generated C code to support module_files from user supplied import paths.

The examples in #1863 (comment) and #1863 (comment) do not need import path since the module py_module.py is present in the current execution directory which is by default accessible.

[Shaikh-Ubaid]

PyUnicode_AsUTF8() returns a const char *. We are casting it here to char *.

[Shaikh-Ubaid]

I think we should create a copy of the const char* string. I think/guess we can support it in another PR.

[certik]

I would call it a header, I think it's more intuitive what it means.

[certik]

The @pythoncall(module="py_module") usage is fine.

[Shaikh-Ubaid]

Got it. I reverted it back to using header. I added support for both the keywords header and module.

[certik]

This looks great, thanks @Shaikh-Ubaid for working on this. Let's figure out how to test this, probably we need to add some new "label" into the integration_tests/CMakeLists.txt for this.

[Shaikh-Ubaid]

Ready.

[certik]

Does this work with cpython? We should try to implement it, to make sure all the syntax works.

[certik]

we have to import pythoncall from here, and implement it in cpython mode.

[certik]

I think that's fine for now. Conda is currently required.

[certik]

I think this looks good. I would still get cpython working. That way the test will be cpython and c, which is fine. We can add llvm later. It is to ensure that the syntax+semantics truly works in both CPython and LPython.

Regarding how thing are handled at the ASR level, @czgdp1807 can you please review this? I think we discussed that we would insert explicit type conversions in ASR between our integer and BindPython integer. However I think we can also do that in later PRs.

[certik]

I think this looks good now, good enough to merge.

We can then iteratively improve this.