Trouble Installing Latest Version of python-flint via pip

[michaelstepniczka]

Hello,

I hope this is the correct location to ask about such an issue. I have been working with pip install --upgrade python-flint==0.3.0 without problem, and recently tried to install the latest version via pip install python-flint after uninstalling the above version. When doing so, I run into the following error message:

  Collecting python-flint
  Using cached python-flint-0.6.0.tar.gz (3.1 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: python-flint
  Building wheel for python-flint (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for python-flint (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [178 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-aarch64-cpython-311
      creating build/lib.linux-aarch64-cpython-311/flint
      copying src/flint/__init__.py -> build/lib.linux-aarch64-cpython-311/flint
      creating build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/__init__.py -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      creating build/lib.linux-aarch64-cpython-311/flint/flint_base
      copying src/flint/flint_base/__init__.py -> build/lib.linux-aarch64-cpython-311/flint/flint_base
      creating build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/__init__.py -> build/lib.linux-aarch64-cpython-311/flint/types
      creating build/lib.linux-aarch64-cpython-311/flint/functions
      copying src/flint/functions/__init__.py -> build/lib.linux-aarch64-cpython-311/flint/functions
      creating build/lib.linux-aarch64-cpython-311/flint/utils
      copying src/flint/utils/flint_exceptions.py -> build/lib.linux-aarch64-cpython-311/flint/utils
      copying src/flint/utils/__init__.py -> build/lib.linux-aarch64-cpython-311/flint/utils
      creating build/lib.linux-aarch64-cpython-311/flint/test
      copying src/flint/test/__init__.py -> build/lib.linux-aarch64-cpython-311/flint/test
      copying src/flint/test/test.py -> build/lib.linux-aarch64-cpython-311/flint/test
      copying src/flint/test/__main__.py -> build/lib.linux-aarch64-cpython-311/flint/test
      running egg_info
      writing src/python_flint.egg-info/PKG-INFO
      writing dependency_links to src/python_flint.egg-info/dependency_links.txt
      writing top-level names to src/python_flint.egg-info/top_level.txt
      reading manifest file 'src/python_flint.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching 'CHANGES'
      adding license file 'LICENSE'
      writing manifest file 'src/python_flint.egg-info/SOURCES.txt'
      copying src/flint/pyflint.c -> build/lib.linux-aarch64-cpython-311/flint
      copying src/flint/pyflint.pxd -> build/lib.linux-aarch64-cpython-311/flint
      copying src/flint/pyflint.pyx -> build/lib.linux-aarch64-cpython-311/flint
      copying src/flint/flintlib/__init__.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/acb.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/acb_calc.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/acb_dft.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/acb_dirichlet.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/acb_elliptic.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/acb_hypgeom.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/acb_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/acb_modular.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/acb_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/arb.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/arb_fmpz_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/arb_hypgeom.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/arb_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/arb_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/arf.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/arith.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/bernoulli.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/dirichlet.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/flint.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpq.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpq_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpq_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_factor.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_lll.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_mod.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_mod_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_mod_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_mod_poly_factor.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_mod_vec.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_mpoly.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_poly_factor.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/fmpz_vec.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/mag.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/mpoly.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/nmod.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/nmod_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/nmod_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/nmod_poly_factor.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/nmod_vec.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/partitions.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flintlib/ulong_extras.pxd -> build/lib.linux-aarch64-cpython-311/flint/flintlib
      copying src/flint/flint_base/flint_base.c -> build/lib.linux-aarch64-cpython-311/flint/flint_base
      copying src/flint/flint_base/flint_base.pxd -> build/lib.linux-aarch64-cpython-311/flint/flint_base
      copying src/flint/flint_base/flint_base.pyx -> build/lib.linux-aarch64-cpython-311/flint/flint_base
      copying src/flint/flint_base/flint_context.c -> build/lib.linux-aarch64-cpython-311/flint/flint_base
      copying src/flint/flint_base/flint_context.pxd -> build/lib.linux-aarch64-cpython-311/flint/flint_base
      copying src/flint/flint_base/flint_context.pyx -> build/lib.linux-aarch64-cpython-311/flint/flint_base
      copying src/flint/types/acb.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb_mat.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb_mat.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb_poly.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb_poly.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb_series.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb_series.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/acb_series.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb_mat.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb_mat.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb_poly.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb_poly.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb_series.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb_series.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arb_series.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arf.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arf.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/arf.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/dirichlet.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/dirichlet.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/dirichlet.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq_mat.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq_mat.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq_poly.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq_poly.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq_series.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq_series.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpq_series.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mat.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mat.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mod.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mod.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mod.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mod_mat.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mod_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mod_mat.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mod_poly.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mod_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mod_poly.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mpoly.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mpoly.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_mpoly.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_poly.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_poly.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_series.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_series.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/fmpz_series.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod_mat.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod_mat.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod_mat.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod_poly.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod_poly.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod_poly.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod_series.c -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod_series.pxd -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/types/nmod_series.pyx -> build/lib.linux-aarch64-cpython-311/flint/types
      copying src/flint/functions/showgood.c -> build/lib.linux-aarch64-cpython-311/flint/functions
      copying src/flint/functions/showgood.pyx -> build/lib.linux-aarch64-cpython-311/flint/functions
      copying src/flint/utils/conversion.pxd -> build/lib.linux-aarch64-cpython-311/flint/utils
      copying src/flint/utils/typecheck.pxd -> build/lib.linux-aarch64-cpython-311/flint/utils
      running build_ext
      building 'flint.pyflint' extension
      creating build/temp.linux-aarch64-cpython-311
      creating build/temp.linux-aarch64-cpython-311/src
      creating build/temp.linux-aarch64-cpython-311/src/flint
      gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/cytools/cytools-venv/include -I/usr/local/include/python3.11 -c src/flint/pyflint.c -o build/temp.linux-aarch64-cpython-311/src/flint/pyflint.o
      src/flint/pyflint.c:1254:10: fatal error: flint/mag.h: No such file or directory
       1254 | #include "flint/mag.h"
            |          ^~~~~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for python-flint
Failed to build python-flint
ERROR: Could not build wheels for python-flint, which is required to install pyproject.toml-based projects
WARNING: There was an error checking the latest version of pip.
Note: you may need to restart the kernel to use updated packages.

I apologize if this is the wrong place to ask such a question, or if this is a personal issue rather than an issue with the code. Thank you for the help!

[oscarbenjamin]

I hope this is the correct location to ask about such an issue.

Indeed it is.

Okay, so there are a few issues here. The first is that binary wheels are provided for many common platforms but not for Linux aarch64 (gh-105) which I'm assuming is your platform.

Are you on Linux aarch64?

Those wheels will hopefully be added soon, but we're basically waiting for GitHub Actions to provide the runners that could build and test them.

The second issue is that building from source with the current release (0.6.0) uses setuptools which does not check the Flint version.

Can you try installing the latest prerelease with one of:

pip install --pre python-flint
pip install python-flint==0.7.0a4

I would be interested to know what output you see but I hope it will be something like:

Error: Flint >= 3.0 is required.

The fact that it can check the Flint version is because the build system has changed from setuptools to meson and meson can check the version and fail the build. Oh, actually no it won't because we're not checking it here:

python-flint/meson.build

Line 10 in 162ded1

flint_dep = dependency('flint')

It should be something like:

flint_dep = dependency('flint', version: ['>=3.0', '<3.2']

Probably 3.2 is already compatible (we have a CI check against Flint main branch). I'm not sure how to manage an upper cap on the version here. When 3.1 came out it was not compatible (prior versions of python-flint would have failed to build). I expect that the same would happen with 3.2 because it only takes one small change to break the build so realistically each new 3.x version of Flint probably requires a new release of python-flint.

Going forwards I would like to keep 3.0 as the minimum supported Flint version for as long as possible but there were big changes in 3.0 and it didn't make sense to try to support both Flint < 3.0 and Flint >= 3.0 at the same time.

I think that then is the actual problem although I may be wrong. You need Flint >= 3.0 to build python-flint since python-flint 0.5.0. The changelog in the README notes supported version changes.

Flint 3.0 was released about 18 months ago I think? So it's pretty new. You either need to build Flint from source or you can maybe get it in a distro binary.

The only distro version I know of that provides Flint 3.0 is Ubuntu 24.04 (released a few months ago). On that distro you can install Flint 3.0.1 with:

sudo apt-get install libflint-dev

That is tested in CI although not on aarch64 (no CI runners available):

python-flint/.github/workflows/buildwheel.yml

Lines 123 to 139 in 162ded1

	# On new enough Ubuntu we can build against the system deb.
	test_pip_flint_deb:
	name: Build on ${{ matrix.os }}
	runs-on: ${{ matrix.os }}
	strategy:
	fail-fast: false
	matrix:
	os: [ubuntu-24.04]
	steps:
	- uses: actions/checkout@v4
	- uses: actions/setup-python@v5
	with:
	python-version: '3.12'
	- run: sudo apt-get update
	- run: sudo apt-get install libflint-dev
	- run: pip install .
	- run: python -m flint.test --verbose

[michaelstepniczka]

Thank you for the quick reply! Yes, I am on Linux AArch64.

Running pip install python-flint==0.7.0a4 leads to the following error block:

Collecting python-flint==0.7.0a4
  Downloading python_flint-0.7.0a4.tar.gz (250 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 250.5/250.5 kB 1.5 MB/s eta 0:00:00a 0:00:01
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [25 lines of output]
      + meson setup /tmp/pip-install-_k181ju0/python-flint_3bd91e3780e54c6499124c65fceb4dac /tmp/pip-install-_k181ju0/python-flint_3bd91e3780e54c6499124c65fceb4dac/.mesonpy-a16b0mqj -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/tmp/pip-install-_k181ju0/python-flint_3bd91e3780e54c6499124c65fceb4dac/.mesonpy-a16b0mqj/meson-python-native-file.ini
      The Meson build system
      Version: 1.5.1
      Source dir: /tmp/pip-install-_k181ju0/python-flint_3bd91e3780e54c6499124c65fceb4dac
      Build dir: /tmp/pip-install-_k181ju0/python-flint_3bd91e3780e54c6499124c65fceb4dac/.mesonpy-a16b0mqj
      Build type: native build
      Project name: python-flint
      Project version: undefined
      C compiler for the host machine: cc (gcc 10.2.1 "cc (Debian 10.2.1-6) 10.2.1 20210110")
      C linker for the host machine: cc ld.bfd 2.35.2
      Cython compiler for the host machine: cython (cython 3.0.11)
      Host machine cpu family: aarch64
      Host machine cpu: aarch64
      Program python found: YES (/home/cytools/cytools-venv/bin/python3)
      Found pkg-config: YES (/usr/bin/pkg-config) 0.29.2
      Run-time dependency python found: YES 3.11
      Run-time dependency gmp found: YES 6.2.1
      Run-time dependency mpfr found: YES 4.1.0
      Found CMake: /usr/bin/cmake (3.18.4)
      WARNING: CMake Toolchain: Failed to determine CMake compilers state
      Run-time dependency flint found: NO (tried pkgconfig and cmake)
      
      ../meson.build:10:12: ERROR: Dependency "flint" not found, tried pkgconfig and cmake
      
      A full log can be found at /tmp/pip-install-_k181ju0/python-flint_3bd91e3780e54c6499124c65fceb4dac/.mesonpy-a16b0mqj/meson-logs/meson-log.txt
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
WARNING: There was an error checking the latest version of pip.
Note: you may need to restart the kernel to use updated packages.

I also tried installing version 0.5.0, as you suggested this may be at the boundary of where the problem is, relating to the version of Flint, and ran into the following error message:

Collecting python-flint==0.5.0
  Downloading python-flint-0.5.0.tar.gz (3.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 4.8 MB/s eta 0:00:0000:0100:010m
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [96 lines of output]
      /tmp/pip-install-7u9cslrv/python-flint_117b229fbf80434eaad614fae15ad948/setup.py:12: DeprecationWarning:
      
        `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
        of the deprecation of `distutils` itself. It will be removed for
        Python >= 3.12. For older Python versions it will remain present.
        It is recommended to use `setuptools < 60.0` for those Python versions.
        For more details, see:
          https://numpy.org/devdocs/reference/distutils_status_migration.html
      
      
        from numpy.distutils.system_info import default_include_dirs, default_lib_dirs
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
                  other = int(other)
              if typecheck(other, int):
                  if other < 0:
                      raise ValueError("negative shift count")
                  u = fmpz.__new__(fmpz)
                  fmpz_mul_2exp((<fmpz>u).val, self.val, other)
                                                  ^
      ------------------------------------------------------------
      
      src/flint/types/fmpz.pyx:429:45: Cannot convert Python object to 'fmpz_struct *'
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
                  other = int(other)
              if typecheck(other, int):
                  if other < 0:
                      raise ValueError("negative shift count")
                  u = fmpz.__new__(fmpz)
                  fmpz_fdiv_q_2exp((<fmpz>u).val, self.val, other)
                                                     ^
      ------------------------------------------------------------
      
      src/flint/types/fmpz.pyx:452:48: Cannot convert Python object to 'fmpz_struct *'
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
              cdef int ttype = FMPZ_UNKNOWN
              ttype = fmpz_set_any_ref(tval, other)
              if ttype == FMPZ_UNKNOWN:
                  return NotImplemented
              u = fmpz.__new__(fmpz)
              fmpz_and((<fmpz>u).val, self.val, tval)
                                         ^
      ------------------------------------------------------------
      
      src/flint/types/fmpz.pyx:475:36: Cannot convert Python object to 'fmpz_struct *'
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
              cdef int ttype = FMPZ_UNKNOWN
              ttype = fmpz_set_any_ref(tval, other)
              if ttype == FMPZ_UNKNOWN:
                  return NotImplemented
              u = fmpz.__new__(fmpz)
              fmpz_xor((<fmpz>u).val, self.val, tval)
                                         ^
      ------------------------------------------------------------
      
      src/flint/types/fmpz.pyx:539:36: Cannot convert Python object to 'fmpz_struct *'
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-7u9cslrv/python-flint_117b229fbf80434eaad614fae15ad948/setup.py", line 130, in <module>
          ext_modules=cythonize(ext_modules, compiler_directives=compiler_directives),
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1115, in cythonize
          cythonize_one(*args)
        File "/home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1238, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: src/flint/types/fmpz.pyx
      Compiling src/flint/pyflint.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/libc/string.pxd.
      Compiling src/flint/types/fmpz.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/version.pxd.
      Compiling src/flint/types/fmpz_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/type.pxd.
      Compiling src/flint/types/fmpq_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/type.pxd.
      Compiling src/flint/types/nmod_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/type.pxd.
      Compiling src/flint/types/arf.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/float.pxd.
      Compiling src/flint/types/arb.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/float.pxd.
      Compiling src/flint/types/arb_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/type.pxd.
      Compiling src/flint/types/acb.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/complex.pxd.
      Compiling src/flint/types/acb_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/type.pxd.
      Compiling src/flint/types/acb_series.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/libc/string.pxd.
      Compiling src/flint/types/fmpz_mpoly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/version.pxd.
      Compiling src/flint/types/fmpz_mod.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/libc/string.pxd.
      Compiling src/flint/types/fmpz_mod_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/type.pxd.
      [ 1/14] Cythonizing src/flint/types/acb.pyx
      [ 2/14] Cythonizing src/flint/types/arb.pyx
      [ 3/14] Cythonizing src/flint/types/arf.pyx
      [ 4/14] Cythonizing src/flint/types/fmpz.pyx
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
WARNING: There was an error checking the latest version of pip.
Note: you may need to restart the kernel to use updated packages.

However, when I tried to install version 0.4.4, I ran into a similar error to the 0.5.0 message:

Collecting python-flint==0.4.4
  Downloading python-flint-0.4.4.tar.gz (2.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.7/2.7 MB 6.2 MB/s eta 0:00:0000:0100:010m
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [94 lines of output]
      /tmp/pip-install-4ll38jiy/python-flint_d84bae8af9f4412a84d7674c07c190db/setup.py:9: DeprecationWarning:
      
        `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
        of the deprecation of `distutils` itself. It will be removed for
        Python >= 3.12. For older Python versions it will remain present.
        It is recommended to use `setuptools < 60.0` for those Python versions.
        For more details, see:
          https://numpy.org/devdocs/reference/distutils_status_migration.html
      
      
        from numpy.distutils.system_info import default_include_dirs, default_lib_dirs
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
                  other = int(other)
              if typecheck(other, int):
                  if other < 0:
                      raise ValueError("negative shift count")
                  u = fmpz.__new__(fmpz)
                  fmpz_mul_2exp((<fmpz>u).val, self.val, other)
                                                  ^
      ------------------------------------------------------------
      
      src/flint/types/fmpz.pyx:420:45: Cannot convert Python object to 'fmpz_struct *'
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
                  other = int(other)
              if typecheck(other, int):
                  if other < 0:
                      raise ValueError("negative shift count")
                  u = fmpz.__new__(fmpz)
                  fmpz_fdiv_q_2exp((<fmpz>u).val, self.val, other)
                                                     ^
      ------------------------------------------------------------
      
      src/flint/types/fmpz.pyx:443:48: Cannot convert Python object to 'fmpz_struct *'
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
              cdef int ttype = FMPZ_UNKNOWN
              ttype = fmpz_set_any_ref(tval, other)
              if ttype == FMPZ_UNKNOWN:
                  return NotImplemented
              u = fmpz.__new__(fmpz)
              fmpz_and((<fmpz>u).val, self.val, tval)
                                         ^
      ------------------------------------------------------------
      
      src/flint/types/fmpz.pyx:466:36: Cannot convert Python object to 'fmpz_struct *'
      
      Error compiling Cython file:
      ------------------------------------------------------------
      ...
              cdef int ttype = FMPZ_UNKNOWN
              ttype = fmpz_set_any_ref(tval, other)
              if ttype == FMPZ_UNKNOWN:
                  return NotImplemented
              u = fmpz.__new__(fmpz)
              fmpz_xor((<fmpz>u).val, self.val, tval)
                                         ^
      ------------------------------------------------------------
      
      src/flint/types/fmpz.pyx:530:36: Cannot convert Python object to 'fmpz_struct *'
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-4ll38jiy/python-flint_d84bae8af9f4412a84d7674c07c190db/setup.py", line 126, in <module>
          ext_modules=cythonize(ext_modules, compiler_directives=compiler_directives),
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1115, in cythonize
          cythonize_one(*args)
        File "/home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Build/Dependencies.py", line 1238, in cythonize_one
          raise CompileError(None, pyx_file)
      Cython.Compiler.Errors.CompileError: src/flint/types/fmpz.pyx
      Compiling src/flint/pyflint.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/libc/string.pxd.
      Compiling src/flint/types/fmpz.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/version.pxd.
      Compiling src/flint/types/fmpz_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/version.pxd.
      Compiling src/flint/types/fmpq_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/type.pxd.
      Compiling src/flint/types/nmod_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/type.pxd.
      Compiling src/flint/types/arf.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/float.pxd.
      Compiling src/flint/types/arb.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/float.pxd.
      Compiling src/flint/types/arb_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/type.pxd.
      Compiling src/flint/types/acb.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/complex.pxd.
      Compiling src/flint/types/acb_poly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/type.pxd.
      Compiling src/flint/types/acb_series.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/libc/string.pxd.
      Compiling src/flint/types/fmpz_mpoly.pyx because it depends on /home/cytools/cytools-venv/lib/python3.11/site-packages/Cython/Includes/cpython/version.pxd.
      [ 1/12] Cythonizing src/flint/types/acb.pyx
      [ 2/12] Cythonizing src/flint/types/arb.pyx
      [ 3/12] Cythonizing src/flint/types/arf.pyx
      [ 4/12] Cythonizing src/flint/types/fmpz.pyx
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
WARNING: There was an error checking the latest version of pip.
Note: you may need to restart the kernel to use updated packages.

In all three cases the error was error: metadata-generation-failed, which may indicate there is something beyond the version of Flint being used? I'm not sure if this would be due to the AArch64 architecture. I hope these error messages are useful!

[oscarbenjamin]

Running pip install python-flint==0.7.0a4 leads to the following error block:

Run-time dependency flint found: NO (tried pkgconfig and cmake)

  ../meson.build:10:12: ERROR: Dependency "flint" not found, tried pkgconfig and cmake

What version of Flint do you have?

My guess it is that it is an old enough version that it just doesn't provide a flint.pc pkgconfig file so meson just doesn't find it at all. It would be nice to be able to detect the version still and give a better error message. We would have to fall back on using meson's find_library though which is unable to check the version so I don't think that can be improved. In future when the minimum version increases from Flint 3.0 we will be able to give a good error message for Flint versions that are too old for python-flint but still new enough to have flint.pc.

However, when I tried to install version 0.4.4, I ran into a similar error to the 0.5.0 message:

I think that with these versions of python-flint the issues you see are to do with not have the correct versions of things like cython. Now python-flint has a pyproject.toml:

python-flint/pyproject.toml

Lines 1 to 3 in 162ded1

	[build-system]
	requires = ["meson-python", "cython"]
	build-backend = "mesonpy"

We should probably put a constraint on the Cython version. At least Cython >= 3.0 is needed for current python-flint I think.

If you want to try building those previous versions then you would need to use something like:

pip install numpy setuptools cython==<some version>
pip install --no-build-isolation python-flint==0.4.4

Unfortunately I don't remember what the necessary Cython version was back in 0.4.4...

You can see the list of Cython releases here:
https://pypi.org/project/Cython/#history

In all three cases the error was error: metadata-generation-failed, which may indicate there is something beyond the version of Flint being used? I'm not sure if this would be due to the AArch64 architecture. I hope these error messages are useful!

They are useful. No they are not to do with aarch64. The error messages are hard to understand because you have pip calling itself in an isolated environment that then calls cython in another subprocess and so you get this Russian doll error message inside error message situation. If you know how to parse the error messages then the relevant parts are:

For Flint 0.7.0a4 the error message is:

      ../meson.build:10:12: ERROR: Dependency "flint" not found, tried pkgconfig and cmake

This is because you don't have a new enough version of Flint. In fact your Flint is too old for meson to find it (no flint.pc) let alone report the version failure.

For Flint 0.4.4 and 0.5.0 the error message is:

      src/flint/types/fmpz.pyx:530:36: Cannot convert Python object to 'fmpz_struct *'

This is Cython failing to compile python-flint's cython code. I think that this is because an older version of Cython is being used or possibly it is that newer Cython does not work with older python-flint or something. Basically you don't have the right version of cython for that particular version of python-flint. If you were to fix it then I would expect that 0.5.0 would fail with the same missing header error as 0.6.0 but hopefully 0.4.4 would work (although that still requires some minimum Flint version and you haven't said what version of Flint you have).

All of these things are going to be much improved in 0.7.0 with meson as the build system and pyproject.toml with version constraints. The pyproject.toml will ensure correct versions of build requirements like Cython and the meson build system will properly detect required C compilers and version of Flint. That should mean that you either get a clear error for unsupported Flint versions (or missing C compilers) or that you get a successful build.

However to use any python-flint newer than 0.4.4 you will need to use at least Flint 3.0. You haven't said what Flint version you have but I am fairly sure that is the source of the problem that prevents that latest version of python-flint from compiling on your machine. So the solution to your problem is either:

1. Install Flint >= 3.0 and then pip install python-flint==0.6.0 should work.
2. Figure out the right version of Cython and install python-flint==0.4.4.

[oscarbenjamin]

In parallel I have been making a pull request (gh-194) that will check the Flint version properly so that for 0.7.0 only tested versions of Flint will be accepted and hopefully the error message will be improved to look like this one:

   ../meson.build:10:12:
   ERROR: Dependency lookup for flint with method 'pkgconfig' failed:
   Invalid version, need 'flint' ['<3.2'] found '3.2.0'.

[michaelstepniczka]

What version of Flint do you have?

I realize I was actually able to run Flint without going to the website to download it because of another piece of software I use, CYTools. As Flint was installed in my instillation of CYTools, I'm not precisely sure what underlying version I am using is; is there a way to query for this in python? Running print(flint.__version__) simply tells me module 'flint' has no attribute '__version__'. After installing CYTools, I was able to run import flint without problem, and only see that the version of python-flint is 0.3.0, but don't see the underlying version of Flint.

Edit: in pip list, the version of python-flint is 0.3.0.

Given the unique function names and such within Flint, as well as minimal outside dependencies, I would imagine that installing the newer version of Flint would not affect the functionality of the CYTools package (outside of new function names/usages). However, I'm unsure as to whether I would need to find where Flint was downloaded within CYTools, uninstall it there, redownload a newer version, and proceed, or if I could just go to the Flint website and download the newest version available.

At least Cython >= 3.0 is needed for the current python-flint.

print(cython.__version__) return 0.29.34. However, after uninstalling and reinstalling Cython, with the uninstall message:

Found existing installation: Cython 0.29.34
Uninstalling Cython-0.29.34:
  Successfully uninstalled Cython-0.29.34

and the reinstall message:

Collecting cython
  Using cached Cython-3.0.11-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.4 MB)
Installing collected packages: cython
Successfully installed cython-3.0.11
WARNING: There was an error checking the latest version of pip.
Note: you may need to restart the kernel to use updated packages.

the version printed via print(cython.__version__) still returns 0.29.34. I therefore hope that I am using the correct version of Cython.

Edit: in pip list, the version of Cython is 3.0.11.

Thank you for helping answer these questions! When I know how, I will try to install a newer version of Flint to see if pip install python-flint==0.6.0 then works! As an aside, will there be other problems with running this install because of the aarch64 architecture and the lack of binary wheels?

[oscarbenjamin]

Running print(flint.__version__) simply tells me module 'flint' has no attribute '__version__'

There is a way to get the Flint version from python-flint but er... only in python-flint 0.6.0 :)

In [1]: import flint

In [2]: flint.__version__
Out[2]: '0.7.0a4'

In [3]: flint.__FLINT_VERSION__
Out[3]: '3.1.0'

In [4]: flint.__FLINT_RELEASE__
Out[4]: 30100

I would imagine that installing the newer version of Flint would not affect the functionality of the CYTools package

No, it will completely break I think if you install Flint >= 3.0 but CYTools expects some older version of Flint.

This does complicate things. In principle there should be a way to have pip and meson-python build their own version of Flint and bundle it into the Python installation isolated from the rest of the system. This was discussed here but no conclusion was reached:
mesonbuild/meson-python#410

Otherwise you would need some way to install a separate version of Flint and sort of activate it somehow when you want to use python-flint which can be done with environment variables but is messy.

I think that at this point you are probably best to stick with python-flint 0.3.0 until the wheels are ready for aarch64. The newest version of python-flint that you can have is 0.4.4 but the changes from 0.3.0 to 0.4.4 are unlikely to be significant for you. A few bugs were fixed, especially so that it would work on Windows, but most other changes were just adding a lot of tests and adding binary wheels.

the version printed via print(cython.__version__) still returns 0.29.34. I therefore hope that I am using the correct version of Cython.

Edit: in pip list, the version of Cython is 3.0.11.

You have multiple installations of Python. When you run print(cython.__version__) (from a notebook?) you are using one Python installation. When you run pip list you are getting the pip from a different Python installation.

As an aside, will there be other problems with running this install because of the aarch64 architecture and the lack of binary wheels?

This one I can't answer. I don't have any Linux aarch64 machine to test on and no way to test in CI. I assume that it works fine because there are no bug reports, just a request to add binary wheels (gh-105).

[michaelstepniczka]

Thank you for all your help! As you suggest, I will stick with python-flint 0.3.0 until the wheels are ready for aarch64. Is there any known or rumored timeline for this?

The main use-case I have been using Flint for is for univariate polynomial multiplication with rational coefficients. I have two polynomials, p1 and p2, for which I keep terms up to O(x^5000), and then wish to find their product up to O(x^5000). Importantly, we require the exact rational values of these coefficients.

Flint has been much faster than other packages such as SymPy, though I noticed in the most recent versions of Flint, that there are a few more options for potential speedups; namely, the mullow method which would keep only terms for this truncation, as well as multithreading capabilities. These polynomials can have more and more 'complicated' expressions for higher coefficients, and so I would naively assume that the mullow method and multithreading could give some sort of substantial speedup.

When you run print(cython.version) (from a notebook?)

I was indeed playing around with Flint in a notebook. I imagine this is not an issue.

You have multiple installations of Python.

Would this lead to problems if the multiple installations of Python are distinct?

Is it standard to keep the issue open until the aarch64 wheels are ready, or should the issue be marked as resolved?

[oscarbenjamin]

python-flint 0.3.0 until the wheels are ready for aarch64. Is there any known or rumored timeline for this?

We're waiting for GitHub Actions to provide CI runners. They have said something vague like "maybe by the end of the year". Then if there are no problems with it adding the platform to the wheel-building matrix is not difficult.

Flint has been much faster than other packages such as SymPy

The latest version of SymPy (1.13) will use python-flint internally for some things like this but only if you have python-flint>=0.6.0.

Specifically in SymPy 1.13 a univariate Poly with coefficients in ZZ, QQ or GF(p) will actually have a Flint polynomial inside:

In [1]: p = Poly(x**2 + 1)

In [2]: p
Out[2]: Poly(x**2 + 1, x, domain='ZZ')

In [3]: p.rep
Out[3]: DUP_Flint([1, 0, 1], ZZ)

In [4]: p.rep._rep
Out[4]: x^2 + 1

In [5]: type(p.rep._rep)
Out[5]: flint.types.fmpz_poly.fmpz_poly

When python-flint 0.7.0 is released it will also have multivariate polynomials over the same domains and I expect that SymPy 1.14 will use those for multivariate Poly as well.

there are a few more options for potential speedups; namely, the mullow method which would keep only terms for this truncation, as well as multithreading capabilities

I haven't actually looked into how python-flint interacts with Flint's multithreading capabilities. We have this:

In [6]: from flint import ctx

In [7]: ctx.threads
Out[7]: 1

I'm just not sure what that actually means in practice.

As for mullow I'm not sure how much speedup you would reasonably get in your situation. You can use that already though because python-flint's fmpq_series type already uses mullow in 0.3.0 I think.

[oscarbenjamin]

Is it standard to keep the issue open until the aarch64 wheels are ready, or should the issue be marked as resolved?

We have a separate issue for that (gh-105).

I have also opened a PR (gh-194) that adds better checking of build requirements to fix some of the other issues identified above. Hopefully in future someone attempting pip install on a platform that does not have wheels will see better error messages or will not have problems like the wrong Cython version.

I would say that we can close this issue although feel free to open another about adding any particular functionality from Flint that you think would find useful.

[oscarbenjamin]

python-flint's fmpq_series type already uses mullow in 0.3.0 I think.

At least in current Flint this is how you use it:

In [6]: from flint import *

In [7]: ctx.cap = 5

In [8]: x = fmpq_series([0, 1])

In [9]: x
Out[9]: x + O(x^5)

In [10]: x.exp()
Out[10]: 1 + x + 1/2*x^2 + 1/6*x^3 + 1/24*x^4 + O(x^5)

In [11]: x.exp() ** 2
Out[11]: 1 + 2*x + 2*x^2 + 4/3*x^3 + 2/3*x^4 + O(x^5)

I tried setting the cap to 5000 to see what happens:

In [17]: ctx.cap = 5000

In [18]: x = fmpq_series([0, 1])

In [19]: %time xexp = x.exp()
CPU times: user 38.6 ms, sys: 0 ns, total: 38.6 ms
Wall time: 37.7 ms

In [20]: %time xexp2 = 2*xexp
CPU times: user 53.3 ms, sys: 3.92 ms, total: 57.2 ms
Wall time: 56.9 ms

In [21]: %time prod = xexp * xexp2
CPU times: user 4.9 s, sys: 441 ms, total: 5.34 s
Wall time: 5.36 s

In [26]: xexp_poly = fmpq_poly(list(xexp))

In [27]: %time prod_poly = xexp_poly * (2* xexp_poly)
CPU times: user 3.71 s, sys: 313 ms, total: 4.02 s
Wall time: 4.14 s

So it doesn't seem like mullow is much faster in this situation. I think that is because it is actually just computing the full product and discarding the terms.

Using mullow is a lot faster if you have high degree polynomials but only want to keep a low degree part of the product. In your case you want to multiply two 5000 degree polynomials making a 10000 degree polynomial and then discard half of the terms. I expect that it is just faster to do this by using the full optimised fmpq_poly_mul.

[oscarbenjamin]

I haven't actually looked into how python-flint interacts with Flint's multithreading capabilities.

So I got a small speedup (20%) from using 4 threads over 1:

In [18]: ctx.threads = 1

In [19]: %time r = xexp ** 2
CPU times: user 3.63 s, sys: 156 ms, total: 3.79 s
Wall time: 3.77 s

In [20]: ctx.threads = 4

In [21]: %time r = xexp ** 2
CPU times: user 4.87 s, sys: 262 ms, total: 5.14 s
Wall time: 2.98 s

No point in going higher than 4 on this machine but I will certainly test on another machine at some point.

[michaelstepniczka]

Thank you for the advice!

I implemented fmpq_series rather than fmpq_poly for the multiplication, and at 10000 terms, the series multiplication takes 14.6 seconds as opposed to 21.5 seconds. Of course, I had to set the cap ctx.cap = 10000. As an aside, where would I find the documentation for these global parameters? Without setting the cap, the series multiplication had a default cap to O(x**10).

Using mullow is a lot faster if you have high degree polynomials but only want to keep a low degree part of the product. In your case you want to multiply two 5000 degree polynomials making a 10000 degree polynomial and then discard half of the terms. I expect that it is just faster to do this by using the full optimised fmpq_poly_mul.

Unfortunately, I do not see fmpq_poly_mul in python-flint 0.3.0. Thankfully, I will use the series multiplication for the time being.

I achieved a slightly larger speedup setting the threads to 4 over 1, achieving a time of ~10 seconds. I'll test it with a higher thread count later -- I need to reinstall CYTools as I messed around with the packages too much.

As a note which I will work on resolving: a collaborator who is also on an aarch64 machine was able to successfully install python-flint 0.6.0 despite also having CYTools. Part of the problem I was having may have been that CYTools runs on a Docker image, and I was running Jupyter notebooks through this Docker image while also using CYTools. Perhaps if I am in an environment outside of this Docker image and away from CYTools, I can run the updated version of python-flint. I will try this out and let you know!

[oscarbenjamin]

I do not see fmpq_poly_mul

This is the C function that is called internally when you multiply two fmpq_poly like:

In [6]: fmpq_poly([1, 2]) * fmpq_poly([3, 4])
Out[6]: 8*x^2 + 10*x + 3

My suggestion is that at the C level mullow may actually just end up calling this same function anyway if you are not asking for a small number of terms.

Part of the problem I was having may have been that CYTools runs on a Docker image, and I was running Jupyter notebooks through this Docker image while also using CYTools.

Okay, and this explains why you have multiple Python installations: one is inside Docker.

Having just revisited the error messages above I realise that it is not (necessarily) that you have an old version of Flint. Rather the error messages are consistent with simply not having Flint at all. If you do only have it inside a Docker image then that would make total sense.

If that is the case then I would suggest to just build and install Flint 3.1. Then pip install python-flint should get you the latest version no problems.

That being said it sounds like you have a complicated setup so it's hard for me to be sure of anything...

[michaelstepniczka]

I now have a fresh install of CYTools with python-flint 0.3.0 working on it again. Running the same tests, I find fmpq_poly multiplication of 10000 term polynomials takes 18.2 seconds (as opposed to the 21.5 seconds quoted above -- I'm not sure where the time difference came from) whereas fmpq_series multiplication of 10000 terms is completed in 14.0 seconds. After setting the number of threads ctx.threads = 4, the fmpq_poly multiplication takes 14.9 seconds and the fmpq_series computation takes 10.0 seconds. Setting the number of threads ctx.threads = 8 has no substantial time difference; this may be due to Docker only having access to limited resources, as I should have more cpu cores.

Regardless, even with python-flint 0.3.0, this is a wonderful time improvement over my other attempts! I plan to build and install Flint 3.1 as suggested and will send updates as they come. Hopefully this resolves the issues!

Repeating a question from earlier: is there somewhere I should look to find the global attributes such as ctx.cap and ctx.threads in the python version of the code? Tab completing I can find others and call a help command, but the Flint documentation on https://flintlib.org/doc/ shows global attributes in flint.h. Perhaps the answer is simply tab completion and looking at the docstrings to have a python-flint API?