Symbolic API

[certik]

Just like the Array API (#1458) we should have a Symbolic API. Both the Symbolic API and Array API share the same basic principles:

• The API implements the core functionality that is "intrinsic" (must be implemented by the backend)
• If you write a library, say SciPy or SymPy limits, you write it using this core API, and then your library is guaranteed to work no matter what Python expression gets passed to it: for SciPy whether it is a NumPy or CuPy or PyTorch or Jax expression, SciPy will still just work and operate on top of these "backends". The same for SymPy, the limits module would work on top of many of the backends that we will have: SymPy Core, SymEngine.
• LPython also implements both the Array and Symbolic API, so the limit module, or SciPy code should just work. The only requirement is that it must be properly annotated and use the subset that LPython supports, but eventually we will make sure that our subset is "good enough" that people would just use it and get the speed that LPython delivers.

For SymPy in particular this will allow us to gradually move all of SymPy on top of this API, and get the best performance:

• Either in CPython via SymEngine (or other such core libraries): the speed is delivered by the backend, the rest runs in pure Python
• Or with LPython: everything gets compiled and you get the best possible performance, but you are restricted to the LPython subset

For development one can use the pure Python backend (SymPy Core), so SymPy would operate exactly as it does now from the end user perspective, nothing gets broken. This provides an upgrade path for SymPy to use SymEngine for all its operations which is better than our old plan (https://github.com/symengine/symengine/wiki/SymPy-core-upgrade-to-SymEngine). This new plan is simply:

• Figure out the Symbolic API in LPython (Design of SymbolicExpression types #1607)
• Port any parts of SymPy simply by compiling it with LPython, and follow LPython's error messages until it compiles. It's a gradual process and at each step the code still works with regular SymPy, so we can simply be sending PRs as our time allows, they will always pass the SymPy's CI and we can merge them. Once a given file fully compiles with LPython, then we have a 100% guarantee that this file will work with all the backends (LPython, SymEngine, SymPy Core) out of the box. We simply port SymPy file by file.

Exactly the same approach will work for SciPy or any other code that uses NumPy. LPython would guarantee that if it compiles with LPython, the same code will run with NumPy, PyTorch, CuPy, etc., as long as the backend follows the Array API.

Once LPython works with SymPy using this approach, it will also help projects like https://github.com/symforce-org/symforce, which also seem to use SymPy API and SymEngine.