0.4.4 tests fail in PyPy

[isuruf]

test_fmpz...Error in test_fmpz
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/python-flint_1694094201440/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/pypy3.9/site-packages/flint/test/__main__.py", line 36, in run_tests
    test()
  File "/home/conda/feedstock_root/build_artifacts/python-flint_1694094201440/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla/lib/pypy3.9/site-packages/flint/test/test.py", line 141, in test_fmpz
    assert pow(a, flint.fmpz(b), c) == ab_mod_c
TypeError: operands do not support pow()

This used to work in 0.4.2

[oscarbenjamin]

I haven't tested with PyPy at all. What version of PyPy are you using for this?

[isuruf]

PyPy3.9 7.3.12

[oscarbenjamin]

PyPy3.9 7.3.12

The tests all passed for me locally with PyPy3.9 7.3.11.

[oscarbenjamin]

I tried reproducing this in CI in #76 but the wheel build fails under PyPy.

[oscarbenjamin]

the wheel build fails under PyPy.

  + python -m pip wheel /Users/runner/work/python-flint/python-flint --wheel-dir=/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-ck51u289/pp39-macosx_x86_64/built_wheel --no-deps
  Processing /Users/runner/work/python-flint/python-flint
    Preparing metadata (setup.py): started
    Preparing metadata (setup.py): finished with status 'error'
    error: subprocess-exited-with-error
    
    × python setup.py egg_info did not run successfully.
    │ exit code: 1
    ╰─> [16 lines of output]
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 34, in <module>
          File "/Users/runner/work/python-flint/python-flint/setup.py", line 9, in <module>
            from numpy.distutils.system_info import default_include_dirs, default_lib_dirs
          File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-ck51u289/pp39-macosx_x86_64/build/venv/lib/pypy3.9/site-packages/numpy/__init__.py", line 139, in <module>
            from . import core
          File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-ck51u289/pp39-macosx_x86_64/build/venv/lib/pypy3.9/site-packages/numpy/core/__init__.py", line 23, in <module>
            from . import multiarray
          File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-ck51u289/pp39-macosx_x86_64/build/venv/lib/pypy3.9/site-packages/numpy/core/multiarray.py", line 86, in <module>
            def empty_like(prototype, dtype=None, order=None, subok=None, shape=None):
          File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-ck51u289/pp39-macosx_x86_64/build/venv/lib/pypy3.9/site-packages/numpy/core/overrides.py", line 178, in decorator
            return array_function_dispatch(
          File "/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/cibw-run-ck51u289/pp39-macosx_x86_64/build/venv/lib/pypy3.9/site-packages/numpy/core/overrides.py", line 160, in decorator
            public_api = _ArrayFunctionDispatcher(dispatcher, implementation)
        AttributeError: 'builtin_function_or_method' object has no attribute '__qualname__'

[oscarbenjamin]

PyPy3.9 7.3.12

The tests all passed for me locally with PyPy3.9 7.3.11.

I'm a bit stuck here without being able to reproduce this.

[isuruf]

Do you have a local conda installation? If so, I can give instructions on how to reproduce, else I can try to set it up on CI.

Note that the tests fail on all three OSes at conda-forge/python-flint-feedstock#31

[oscarbenjamin]

I don't even want to try to fix this until we have CI that can reproduce the problem. If we don't test PyPy in CI then I would rather just say that it is not supported.

[oscarbenjamin]

I have just tested locally that I can build a wheel and install from it and all tests are passing.

This is with pypy-3.9-7.3.11 which is the newest version available from pyenv.

If the difference is pypy-3.9-7.3.12 rather than 7.3.11 then that might indicate a PyPy bug rather than a python-flint bug.

The PRs to python-flint in between 0.4.2 and 0.4.4 are:

• Introduce submodules into python-flint #61
• Finish the flint_base submodule, depreciate old roots and factor out more utils #63
• Mod experiment #65
• Bump version to 0.4.3 #69
• Include files in sdist and bump version to 0.4.4 #71

They are all just refactoring or unrelated changes.

The bug reported is with 3-arg pow like:

pow(1, flint.fmpz(2), 3)

This is the code for fmpz.pow in 0.4.2:

python-flint/src/flint/fmpz.pyx

Lines 365 to 412 in b658fd3

	def __pow__(s, t, m):
	cdef fmpz_struct tval[1]
	cdef fmpz_struct mval[1]
	cdef int ttype = FMPZ_UNKNOWN
	cdef int mtype = FMPZ_UNKNOWN
	cdef int success
	u = NotImplemented
	ttype = fmpz_set_any_ref(tval, t)
	if ttype == FMPZ_UNKNOWN:
	return NotImplemented
	if m is None:
	# fmpz_pow_fmpz throws if x is negative
	if fmpz_sgn(tval) == -1:
	if ttype == FMPZ_TMP: fmpz_clear(tval)
	raise ValueError("negative exponent")
	u = fmpz.__new__(fmpz)
	success = fmpz_pow_fmpz((<fmpz>u).val, (<fmpz>s).val, tval)
	if not success:
	if ttype == FMPZ_TMP: fmpz_clear(tval)
	raise OverflowError("fmpz_pow_fmpz: exponent too large")
	else:
	# Modular exponentiation
	mtype = fmpz_set_any_ref(mval, m)
	if mtype != FMPZ_UNKNOWN:
	if fmpz_is_zero(mval):
	if ttype == FMPZ_TMP: fmpz_clear(tval)
	if mtype == FMPZ_TMP: fmpz_clear(mval)
	raise ValueError("pow(): modulus cannot be zero")
	# The Flint docs say that fmpz_powm will throw if m is zero
	# but it also throws if m is negative. Python generally allows
	# e.g. pow(2, 2, -3) == (2^2) % (-3) == -2. We could implement
	# that here as well but it is not clear how useful it is.
	if fmpz_sgn(mval) == -1:
	if ttype == FMPZ_TMP: fmpz_clear(tval)
	if mtype == FMPZ_TMP: fmpz_clear(mval)
	raise ValueError("pow(): negative modulua not supported")
	u = fmpz.__new__(fmpz)
	fmpz_powm((<fmpz>u).val, (<fmpz>s).val, tval, mval)
	if ttype == FMPZ_TMP: fmpz_clear(tval)
	if mtype == FMPZ_TMP: fmpz_clear(mval)
	return u

This is the code in 0.4.4:

python-flint/src/flint/types/fmpz.pyx

Lines 358 to 405 in 24267a5

	def __pow__(s, t, m):
	cdef fmpz_struct tval[1]
	cdef fmpz_struct mval[1]
	cdef int ttype = FMPZ_UNKNOWN
	cdef int mtype = FMPZ_UNKNOWN
	cdef int success
	u = NotImplemented
	ttype = fmpz_set_any_ref(tval, t)
	if ttype == FMPZ_UNKNOWN:
	return NotImplemented
	if m is None:
	# fmpz_pow_fmpz throws if x is negative
	if fmpz_sgn(tval) == -1:
	if ttype == FMPZ_TMP: fmpz_clear(tval)
	raise ValueError("negative exponent")
	u = fmpz.__new__(fmpz)
	success = fmpz_pow_fmpz((<fmpz>u).val, (<fmpz>s).val, tval)
	if not success:
	if ttype == FMPZ_TMP: fmpz_clear(tval)
	raise OverflowError("fmpz_pow_fmpz: exponent too large")
	else:
	# Modular exponentiation
	mtype = fmpz_set_any_ref(mval, m)
	if mtype != FMPZ_UNKNOWN:
	if fmpz_is_zero(mval):
	if ttype == FMPZ_TMP: fmpz_clear(tval)
	if mtype == FMPZ_TMP: fmpz_clear(mval)
	raise ValueError("pow(): modulus cannot be zero")
	# The Flint docs say that fmpz_powm will throw if m is zero
	# but it also throws if m is negative. Python generally allows
	# e.g. pow(2, 2, -3) == (2^2) % (-3) == -2. We could implement
	# that here as well but it is not clear how useful it is.
	if fmpz_sgn(mval) == -1:
	if ttype == FMPZ_TMP: fmpz_clear(tval)
	if mtype == FMPZ_TMP: fmpz_clear(mval)
	raise ValueError("pow(): negative modulua not supported")
	u = fmpz.__new__(fmpz)
	fmpz_powm((<fmpz>u).val, (<fmpz>s).val, tval, mval)
	if ttype == FMPZ_TMP: fmpz_clear(tval)
	if mtype == FMPZ_TMP: fmpz_clear(mval)
	return u

So unless I've missed something the code for fmpz.__pow__ is unchanged although there was a refactor that moved it to a different file.

The code still passes under CPython in any case.

[isuruf]

If the difference is pypy-3.9-7.3.12 rather than 7.3.11 then that might indicate a PyPy bug rather than a python-flint bug.

I checked with 7.3.11 in conda and the bug is still there

[isuruf]

It is a bug in 7.3.12. Disregard my previous comment.

[oscarbenjamin]

Is there an associated PyPy bug report for this?