Uploading wheels to pypi

FYI I'm working on uploading wheels to pypi over at https://github.com/xoviat/build-python-flint. After some pain, I've managed to build python-flint on windows and will probably have builds for OSX and manylinux as well.

Wheels are done for Windows and Linux. Waiting on OSX due to travis-ci/travis-ci#8552.

[isuruf]

@xoviat, you have to be careful about how you build gmp/mpir when you want to distribute. Here's how I do it, https://github.com/symengine/symengine-wheels/blob/master/config.sh#L24. Note that you have to enable --enable-fat or build for a generic x86 machine instead of ivybridge in the travis machines.

Actually I just reused your build_dependencies script to build it. Does that need modification?

[isuruf]

Yes. You need to add --enable-fat for Linux and add --host=x86_64-darwin-none (not sure) on OSX (--enable-fat for mpir doesn't work on Mac). Or you can switch to gmp instead of mpir can add --enable-fat for Linux and Mac both.

[alisianoi]

Here is a wrapper project I made that builds wheels for python-flint in case anyone needs those:

https://gitlab.com/alisianoi/flint-py

The README.md there describes the build process a bit. The wheels are on PyPI:

https://pypi.org/project/flint-py/

I bumped the version from 0.3.0 to 0.3.5 but all the changes were to the build process

[fredrik-johansson]

@alisianoi Thanks for sharing! Would it make sense to integrate this with python-flint instead of having a separate project? It doesn't make much difference either way as long as it's maintained in some form.

[oscarbenjamin]

I think it would definitely make sense to integrate this with python-flint. Setting this up to compile is not completely trivial so it would be much better if there are precompiled wheels for Windows, OSX and manylinux. They could be built in github actions.

[fredrik-johansson]

@oscarbenjamin Indeed. Unfortunately I don't have much time as I'd like to maintain python-flint, and it'd be awesome if someone else was willing to step up. Not least considering all the new multivariate polynomial stuff (and more) in the latest Flint that hasn't been wrapped in python-flint yet.

[oscarbenjamin]

Okay so maybe reopen this issue and I'll see what I can do.

I've almost managed to build a wheel on OSX (build script is below) but I'm hitting against two problems right now:

1. Although I can install with pip install . and I can build a wheel with pip wheel . when I try to install the wheel like pip install *.whl I get this error:

    ```
    $ pip install python_flint-0.3.0-cp38-cp38-macosx_11_0_x86_64.whl
    ERROR: python_flint-0.3.0-cp38-cp38-macosx_11_0_x86_64.whl is not a supported wheel on this platform.
    ```
   

   I think that's telling me that the name of the wheel specifies a system different from this one somehow (even though I built it here!). I've tried a few renames of the file but haven't managed to get pip to accept it.

2. I'll need to get delocate to be able to bundle flint/arb to make the wheel relocatable but that also doesn't seem to be working and I think it's because there is no package in python-flint (it's just a flint.* extension module). The delocate docs suggest that you need to make a package like:

    ```
    flint/
        __init__.py   # this just does from .flint import *
        flint.so
    ```
   

   Then the shared libraries can be bundled into the package folder:

   https://github.com/matthew-brett/delocate#external-library-is-not-bundled-into-wheel

The script I have so far is:

#!/usr/bin/env bash

set -o errexit

PYTHON=python3.8
FLINTVER=2.7.1
ARBVER=2.19.0

mkdir -p local
PREFIX=$(pwd)/local

curl -O https://www.flintlib.org/flint-$FLINTVER.tar.gz
curl -O -L https://github.com/fredrik-johansson/arb/archive/refs/tags/$ARBVER.tar.gz
mv $ARBVER.tar.gz arb-$ARBVER.tar.gz
git clone https://github.com/fredrik-johansson/python-flint

tar xf flint-$FLINTVER.tar.gz
cd flint-$FLINTVER
  ./configure --prefix=$PREFIX
  make -j4
  make install
cd ..

tar xf arb-$ARBVER.tar.gz
cd arb-$ARBVER
  ./configure --prefix=$PREFIX --with-flint=$PREFIX
  make -j4
  make install
cd ..

$PYTHON -m venv venv
source venv/bin/activate
pip install -U pip wheel
pip install numpy cython==0.27.3
# N.B. bugs in both older and newer Cython versions...

cd python-flint
  pip wheel . --global-option=build_ext \
                --global-option="-I$PREFIX/include" \
                --global-option="-L$PREFIX/lib"
  pip install . --global-option=build_ext \
                --global-option="-I$PREFIX/include" \
                --global-option="-L$PREFIX/lib"
cd ..

python -c 'import flint'

[oscarbenjamin]

Okay I've got this working to make some usable OSX wheels. Changes needed in python_flint are:

# Move src to src/flint
mv src flint
mkdir src
mv flint src

Apply this diff in setup.py:

diff --git a/setup.py b/setup.py
index 0a739c9..448466a 100644
--- a/setup.py
+++ b/setup.py
@@ -21,7 +21,7 @@ default_include_dirs += [
 
 ext_modules = [
     Extension(
-        "flint", ["src/pyflint.pyx"],
+        "flint.flint", ["src/flint/pyflint.pyx"],
         libraries=libraries,
         library_dirs=default_lib_dirs,
         include_dirs=default_include_dirs)
@@ -34,6 +34,8 @@ setup(
     name='python-flint',
     cmdclass={'build_ext': build_ext},
     ext_modules=ext_modules,
+    packages=['flint'],
+    package_dir={'': 'src'},
     description='Bindings for FLINT and Arb',
     version='0.3.0',
     url='https://github.com/python-flint/python-flint',

That gives the following package structure:

flint/
    __init__.py
    flint.so

Once those changes are made then this script generates a relocatable OSX wheel on OSX 11.1 with either Python 3.7 or 3.8 from python.org installers. It does not work with python 3.9. I think that is because a newer cython version is needed but the newer cython has a bug that doesn't work for python_flint (#13)

EDIT (the changes above were made in #14 so this script works directly with master now):

#!/usr/bin/env bash

set -o errexit

PYTHON=python3.8
GMPVER=6.2.1
MPFRVER=4.1.0
FLINTVER=2.7.1
ARBVER=2.19.0
PYTHONFLINTVER=0.3.0

mkdir -p local
PREFIX=$(pwd)/local

curl -O https://gmplib.org/download/gmp/gmp-$GMPVER.tar.xz
curl -O https://www.mpfr.org/mpfr-current/mpfr-$MPFRVER.tar.gz
curl -O https://www.flintlib.org/flint-$FLINTVER.tar.gz
curl -O -L https://github.com/fredrik-johansson/arb/archive/refs/tags/$ARBVER.tar.gz
mv $ARBVER.tar.gz arb-$ARBVER.tar.gz
rm -rf python-flint
git clone https://github.com/fredrik-johansson/python-flint

tar xf gmp-$GMPVER.tar.xz
cd gmp-$GMPVER
  ./configure --prefix=$PREFIX --enable-fat --enable-shared --host=x86_64-apple-darwin
  make -j3
  make install
cd ..

tar -xf mpfr-$MPFRVER.tar.gz
cd mpfr-$MPFRVER
  ./configure --prefix=$PREFIX --with-gmp=$PREFIX
  make -j3
  make install
cd ..

tar xf flint-$FLINTVER.tar.gz
cd flint-$FLINTVER
  ./configure --prefix=$PREFIX --with-gmp=$PREFIX --with-mpfr=$PREFIX
  make -j3
  make install
cd ..

tar xf arb-$ARBVER.tar.gz
cd arb-$ARBVER
  ./configure --prefix=$PREFIX --with-flint=$PREFIX --with-gmp=$PREFIX --with-mpfr=$PREFIX
  make -j3
  make install
cd ..

$PYTHON -m venv venv
source venv/bin/activate
pip install -U pip wheel delocate
pip install numpy cython==0.27.3
# N.B. bugs in both older and newer Cython versions...

cd python-flint
  pip wheel .   --global-option=build_ext \
                --global-option="-I$PREFIX/include" \
                --global-option="-L$PREFIX/lib"
  delocate-wheel -w .. *.whl
cd ..

tags="cp37-cp37m cp38-cp38"
osx_from="11_0"
osx_to="10_9"
for tag in $tags; do
  wheelold=python_flint-$PYTHONFLINTVER-$tag-macosx_${osx_from}_x86_64.whl
  wheelnew=python_flint-$PYTHONFLINTVER-$tag-macosx_${osx_to}_x86_64.whl
  if [ -f $wheelold ]; then
    mv $wheelold $wheelnew
  fi
  if [ -f $wheelnew ]; then
    wheelfinal=$wheelnew
  fi
done

echo ------------------------------------------
echo
echo Built wheel: $wheelfinal
echo
echo Link dependencies:
delocate-listdeps $wheelfinal
echo
pip install $wheelfinal
echo
echo Demonstration:
echo
python -c 'import flint; print("(3/2)**2 =", flint.fmpq(3, 2)**2)'
echo
echo Done!
echo
echo ------------------------------------------

[oscarbenjamin]

The changes for python_flint that I described above were made in #14 so the script above builds a wheel from master now.

It doesn't build GMP or MPFR but still works on my system because I have those installed. I guess I should get it to compile them as well then I could see how to get it working in Github Actions.

Ultimately python_flint should probably use cibuildwheels to build wheels on Windows, OSX and Linux but I'll see if I can get a single wheel build working directly first.