spin test fails after spin install

[akhmerov]

I have a meson-python project (https://github.com/akhmerov/python-mumps), and I object the following if I run spin test after spin install:

python-mumps$ spin test
Invoking `build` prior to running tests:
$ meson compile -C build
INFO: autodetecting backend as ninja
INFO: calculating backend command to run: /home/anton/micromamba/envs/py311/bin/ninja -C /home/anton/Bot/coding/python-mumps/build
ninja: Entering directory `/home/anton/Bot/coding/python-mumps/build'
ninja: no work to do.
$ meson install --only-changed -C build --destdir ../build-install
$ export PYTHONPATH="/home/anton/Bot/coding/python-mumps/build-install/usr/lib/python3.12/site-packages"
$ /home/anton/micromamba/envs/py311/bin/python3.12 -P -c 'import mumps'
$ export PYTHONPATH="/home/anton/Bot/coding/python-mumps/build-install/usr/lib/python3.12/site-packages"
$ cd /home/anton/Bot/coding/python-mumps/build-install/usr/lib/python3.12/site-packages
$ /home/anton/micromamba/envs/py311/bin/python3.12 -m pytest --rootdir=/home/anton/Bot/coding/python-mumps/build-install/usr/lib/python3.12/site-packages mumps
================================================================================= test session starts ==================================================================================
platform linux -- Python 3.12.1, pytest-8.0.0, pluggy-1.4.0
Using --randomly-seed=2233073045
rootdir: /home/anton/Bot/coding/python-mumps/build-install/usr/lib/python3.12/site-packages
configfile: ../../../../../pytest.ini
plugins: subtests-0.11.0, flake8-1.1.1, datadir-1.5.0, regressions-2.5.0, cov-4.1.0, flakes-4.0.5, anyio-4.2.0, randomly-3.15.0
collected 0 items / 1 error                                                                                                                                                            

======================================================================================== ERRORS ========================================================================================
______________________________________________________________________ ERROR collecting mumps/tests/test_mumps.py ______________________________________________________________________
import file mismatch:
imported module 'mumps.tests.test_mumps' has this __file__ attribute:
  /home/anton/Bot/coding/python-mumps/mumps/tests/test_mumps.py
which is not the same as the test file we want to collect:
  /home/anton/Bot/coding/python-mumps/build-install/usr/lib/python3.12/site-packages/mumps/tests/test_mumps.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
=============================================================================== short test summary info ================================================================================
ERROR mumps/tests/test_mumps.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=================================================================================== 1 error in 0.25s ===================================================================================

spin test functions properly if I do pip uninstall python-mumps.

[stefanv]

@akhmerov Interesting, I know that editable installs are problematic, but we should be able to get the build install to be preferred over regular pip installs. I'll take a look (are you working on a branch, or is this with python-mumps main?).

You may also be interested in #155

[akhmerov]

Yes, I see this directly on main.

[stefanv]

@akhmerov I am following the python-mumps installation instructions, but running into:

meson.build:51:16: ERROR: Include dir ./../include does not exist.

I tried git submodule update --init, but that did not fetch what was needed. Does it need MUMPS in the parent directory?

[akhmerov]

You need mumps-seq in the conda environment, but I also see that the old master still lingering around and set as default branch in the repo (fixed now). Could it be that you are not on main? The code from your error message is gone on main.

[stefanv]

Yup, looks like master was pulled down by default.

[stefanv]

I don't use conda, is there a pip equivalent of mumps-seq?

[akhmerov]

If you're on ubuntu/debian, I think everything should work if you apt install libmumps-seq-dev. Not sure how one gets binary libraries on other OSes.

[stefanv]

OK, on Fedora I had to install MUMPS-devel and modify the build system to detect the headers:

diff --git a/meson.build b/meson.build
index 7dafcfa..9759251 100644
--- a/meson.build
+++ b/meson.build
@@ -38,6 +38,8 @@ endif
 inc_np = include_directories(incdir_numpy)
 np_dep = declare_dependency(include_directories: inc_np)
 mumps_names = get_option('mumps_names')
+inc_mumps = '/usr/include/MUMPS'
+mumps_dep = declare_dependency(include_directories: inc_mumps)

 mumps_libs = []
 foreach mumps_name : mumps_names
diff --git a/mumps/meson.build b/mumps/meson.build
index d0f73c6..4a04d29 100644
--- a/mumps/meson.build
+++ b/mumps/meson.build
@@ -23,7 +23,7 @@ endforeach

 py.extension_module('_mumps',
     pyx_files,
-    dependencies: [np_dep, mumps_libs],
+    dependencies: [np_dep, mumps_libs, mumps_dep],
     install: true,
     subdir: 'mumps'
 )

It's unclear why they're missing a .pc file.

[stefanv]

Something's also not quite right with the pyproject.toml dependencies declaration, since pip install fails with missing NumPy.

Anyway, after all that I cannot reproduce the failure with the latest spin from main on Python 3.12 :/

[stefanv]

In the above error message, pytest suggests:

HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules

So, maybe try that.

[stefanv]

Here's the problem with the pyproject.toml file:

    "numpy==1.24.4; python_version=='3.11' and platform_python_implementation != 'PyPy'",

Pretty sure that should have a numpy definition for other version of Python too.

[akhmerov]

Thanks for pointing that out, I'll fix that spec in the next versions. That, however, doesn't seem to be the source of the problem and I observed it also on the updated main which just requires numpy. II have now also reproduced the problem in an isolated docker environment:

sudo docker run --rm -it mambaorg/micromamba:1.4.3 bash

micromamba install -y -c conda-forge python cython numpy scipy pytest ninja compilers meson-python mumps-seq git
git clone https://github.com/akhmerov/python-mumps/
cd python-mumps
pip install spin
spin install
spin test

[stefanv]

Can you share the Dockerfile, please?

[akhmerov]

It's mambaorg/micromamba like in the snippet above; I didn't prepare a custom dockerfile. Copypasting the second code block into an interactive shell in the container reproduces the issue.

[stefanv]

Sorry, commented pre-caffeine.