Make binary wheels for all platforms

[rth]

Making binary wheels and uploading them to PyPi, would allow to pip install tslearn without needing a compiler or Cython.

Usually this requires quite a bit of work, see e.g. https://github.com/MacPython/scikit-learn-wheels/. However there is a shortcut with https://github.com/regro/conda-press that might allow generating wheels from conda-forge builds. I have not used it yet personally, but it could be worth a try.

[rtavenar]

OK, we should definitely give it a try.

conda-press option seems very attractive.
Then I guess the release process should be something like:

1. release source archive on PyPI
2. wait for conda-forge to generate builds
3. use conda-press to turn them into wheels locally
4. release wheels on PyPI

[rth]

Yes, we could try it for the current version. But we need to check that generated wheel can be installed correctly.

[rtavenar]

And I guess we should check that on various OS, and if possible on instances that do not have C++ build tools installed, right?

[rtavenar]

OK, I am starting to play with that idea, locally.

With the following code, I can generate all OSX and Windows wheels for tslearn (as soon as the corresponding artifact is available on conda-forge, of course):

TARGET_TSLEARN_VERSION=0.2.1
for OSNAME in osx-64 win-64; 
do 
    for PYTHONVER in py27 py36 py37; 
    do 
        conda press --subdir ${OSNAME} --skip-python --fatten tslearn=${TARGET_TSLEARN_VERSION}=${PYTHONVER}*; 
    done; 
done

I guess this can be a good starting point. I still have issues at the moment when it comes to building linux wheels, which is a pity. I will try to dig that later.

Also, I would say that building fat wheels (ie. that includes the dependencies, if I understood correctly) is a bit of a pity, no? Are there use cases where this would be safer than lighter wheels without dependencies included?

Finally, maybe we should ask the community of tslearn users for help (eg. via Twitter) once we have uploaded the wheels on PyPI so that people can test them, no? But if we do so, maybe the test will be biased (tslearn users are likely to have installed tslearn successfully in the past :P)

@rth: I'd be happy to read your feedback on those points :)

[rtavenar]

A bit more insight into the failure case for Linux. For all python versions, it seems to fail at some point when running the following command (through a subprocess call, but anyway):

install_name_tool -add_rpath @loader_path/../lib /var/folders/q_/n_cbhlk54t7667mq9jl141g80000gr/T/tslearn-0.2.1-py27hc1659b7_060mkj2zw/lib/python2.7/site-packages/tslearn/cycc.so

And if I run the command in terminal, the error message is (once again for all 3 python versions):

error: /Library/Developer/CommandLineTools/usr/bin/install_name_tool: input file: /var/folders/q_/n_cbhlk54t7667mq9jl141g80000gr/T/tslearn-0.2.1-py27hc1659b7_060mkj2zw/lib/python2.7/site-packages/tslearn/cycc.so is not a Mach-O file

Not sure what this means / how to deal with that, but maybe I could post an issue on the conda-press github.

[rth]

Thanks for looking into this @rtavenar !

Are there use cases where this would be safer than lighter wheels without dependencies included?

As far as I understood fat wheels with --skip-python will only include non-python dependencies (e.g. dependencies on C or C++ libraries).

A bit more insight into the failure case for Linux.
maybe I could post an issue on the conda-press github.

Definitely worth reporting. For me Linux and Windows builds fine, while the MacOS fails with,

rewriting RPATH for lib/python2.7/site-packages/tslearn/cycc.so
Traceback (most recent call last):
  File "/home/rth/.miniconda3/envs/conda-press-env/bin/conda-press", line 4, in <module>
    main()
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/conda_press/main.xsh", line 63, in main
    only_pypi=ns.only_pypi
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/conda_press/condatools.xsh", line 866, in artifact_ref_dependency_tree_to_wheels
    only_pypi=only_pypi
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/conda_press/condatools.xsh", line 799, in package_to_wheel
    only_pypi=only_pypi
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/conda_press/condatools.xsh", line 766, in artifact_to_wheel
    wheel.rewrite_rpaths()
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/conda_press/wheel.xsh", line 582, in rewrite_rpaths
    current_rpath = $(patchelf --print-rpath @(fspath)).strip()
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/xonsh/__amalgam__.py", line 22440, in subproc_captured_stdout
    return run_subproc(cmds, captured="stdout")
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/xonsh/__amalgam__.py", line 22424, in run_subproc
    command.end()
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/xonsh/__amalgam__.py", line 15671, in end
    self._end(tee_output=tee_output)
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/xonsh/__amalgam__.py", line 15690, in _end
    self._raise_subproc_error()
  File "/home/rth/.miniconda3/envs/conda-press-env/lib/python3.7/site-packages/xonsh/__amalgam__.py", line 15816, in _raise_subproc_error
    raise subprocess.CalledProcessError(rtn, spec.args, output=self.output)
subprocess.CalledProcessError: Command '['patchelf', '--print-rpath', '/tmp/tslearn-0.2.1-py27heacc8b8_07l8p5764/lib/python2.7/site-packages/tslearn/cycc.so']' returned non-zero exit status 1.

~~will open an issue about it.~~ reported in https://github.com/conda-incubator/conda-press/issues/38

Finally, maybe we should ask the community of tslearn users for help (eg. via Twitter) once we have uploaded the wheels on PyPI so that people can test them, no? But if we do so, maybe the test will be biased (tslearn users are likely to have installed tslearn successfully in the past :P)

Well, it's also possible to just test locally, upload wheels PyPi, announce the release see if there any complaints.

I have checked the python 3.7 wheels, overall it seems to work fine. There are a few test failures though,

platform linux -- Python 3.7.0, pytest-5.2.0, py-1.8.0, pluggy-0.13.0
rootdir: /home/rth/src/tslearn, inifile: setup.cfg
collected 38 items                                                                                                                            

test_barycenters.py ....                                                                                                                [ 10%]
test_clustering.py .....                                                                                                                [ 23%]
test_estimators.py FFFFFFFFF                                                                                                            [ 47%]
test_metrics.py .......                                                                                                                 [ 65%]
test_neighbors.py .                                                                                                                     [ 68%]
test_piecewise.py ...                                                                                                                   [ 76%]
test_shapelets.py F.                                                                                                                    [ 81%]
test_utils.py ...                                                                                                                       [ 89%]
test_variablelength.py ....                                                                                                             [100%]

================================================================== FAILURES ===================================================================
_______________________________________________________ test_all_estimators[estimator0] _______________________________________________________

estimator = ('GlobalAlignmentKernelKMeans', <class 'tslearn.clustering.GlobalAlignmentKernelKMeans'>)

    @pytest.mark.parametrize('estimator', get_estimators('all'))
    def test_all_estimators(estimator):
        """Test all the estimators in tslearn."""
        allow_nan = (hasattr(checks, 'ALLOW_NAN') and
                     _safe_tags(estimator[1](), "allow_nan"))
        if allow_nan:
            checks.ALLOW_NAN.append(estimator[0])
>       check_estimator(estimator[1])

test_estimators.py:178: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_estimators.py:148: in check_estimator
    check_parameters_default_constructible(name, Estimator)
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/estimator_checks.py:2146: in check_parameters_default_constructible
    with ignore_warnings(category=(DeprecationWarning, FutureWarning)):
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:375: in __enter__
    clean_warning_registry()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def clean_warning_registry():
        """Clean Python warning registry for easier testing of warning messages.
    
        We may not need to do this any more when getting rid of Python 2, not
        entirely sure. See https://bugs.python.org/issue4180 and
        https://bugs.python.org/issue21724 for more details.
    
        """
        reg = "__warningregistry__"
        for mod_name, mod in list(sys.modules.items()):
            if hasattr(mod, reg):
>               getattr(mod, reg).clear()
E               AttributeError: 'int' object has no attribute 'clear'

/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:766: AttributeError
_______________________________________________________ test_all_estimators[estimator1] _______________________________________________________

estimator = ('KNeighborsTimeSeriesClassifier', <class 'tslearn.neighbors.KNeighborsTimeSeriesClassifier'>)

    @pytest.mark.parametrize('estimator', get_estimators('all'))
    def test_all_estimators(estimator):
        """Test all the estimators in tslearn."""
        allow_nan = (hasattr(checks, 'ALLOW_NAN') and
                     _safe_tags(estimator[1](), "allow_nan"))
        if allow_nan:
            checks.ALLOW_NAN.append(estimator[0])
>       check_estimator(estimator[1])

test_estimators.py:178: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_estimators.py:148: in check_estimator
    check_parameters_default_constructible(name, Estimator)
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/estimator_checks.py:2146: in check_parameters_default_constructible
    with ignore_warnings(category=(DeprecationWarning, FutureWarning)):
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:375: in __enter__
    clean_warning_registry()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def clean_warning_registry():
        """Clean Python warning registry for easier testing of warning messages.
    
        We may not need to do this any more when getting rid of Python 2, not
        entirely sure. See https://bugs.python.org/issue4180 and
        https://bugs.python.org/issue21724 for more details.
    
        """
        reg = "__warningregistry__"
        for mod_name, mod in list(sys.modules.items()):
            if hasattr(mod, reg):
>               getattr(mod, reg).clear()
E               AttributeError: 'int' object has no attribute 'clear'

/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:766: AttributeError
_______________________________________________________ test_all_estimators[estimator2] _______________________________________________________

estimator = ('KShape', <class 'tslearn.clustering.KShape'>)

    @pytest.mark.parametrize('estimator', get_estimators('all'))
    def test_all_estimators(estimator):
        """Test all the estimators in tslearn."""
        allow_nan = (hasattr(checks, 'ALLOW_NAN') and
                     _safe_tags(estimator[1](), "allow_nan"))
        if allow_nan:
            checks.ALLOW_NAN.append(estimator[0])
>       check_estimator(estimator[1])

test_estimators.py:178: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_estimators.py:148: in check_estimator
    check_parameters_default_constructible(name, Estimator)
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/estimator_checks.py:2146: in check_parameters_default_constructible
    with ignore_warnings(category=(DeprecationWarning, FutureWarning)):
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:375: in __enter__
    clean_warning_registry()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def clean_warning_registry():
        """Clean Python warning registry for easier testing of warning messages.
    
        We may not need to do this any more when getting rid of Python 2, not
        entirely sure. See https://bugs.python.org/issue4180 and
        https://bugs.python.org/issue21724 for more details.
    
        """
        reg = "__warningregistry__"
        for mod_name, mod in list(sys.modules.items()):
            if hasattr(mod, reg):
>               getattr(mod, reg).clear()
E               AttributeError: 'int' object has no attribute 'clear'

/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:766: AttributeError
_______________________________________________________ test_all_estimators[estimator3] _______________________________________________________

estimator = ('LabelCategorizer', <class 'tslearn.utils.LabelCategorizer'>)

    @pytest.mark.parametrize('estimator', get_estimators('all'))
    def test_all_estimators(estimator):
        """Test all the estimators in tslearn."""
        allow_nan = (hasattr(checks, 'ALLOW_NAN') and
                     _safe_tags(estimator[1](), "allow_nan"))
        if allow_nan:
            checks.ALLOW_NAN.append(estimator[0])
>       check_estimator(estimator[1])

test_estimators.py:178: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_estimators.py:148: in check_estimator
    check_parameters_default_constructible(name, Estimator)
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/estimator_checks.py:2146: in check_parameters_default_constructible
    with ignore_warnings(category=(DeprecationWarning, FutureWarning)):
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:375: in __enter__
    clean_warning_registry()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def clean_warning_registry():
        """Clean Python warning registry for easier testing of warning messages.
    
        We may not need to do this any more when getting rid of Python 2, not
        entirely sure. See https://bugs.python.org/issue4180 and
        https://bugs.python.org/issue21724 for more details.
    
        """
        reg = "__warningregistry__"
        for mod_name, mod in list(sys.modules.items()):
            if hasattr(mod, reg):
>               getattr(mod, reg).clear()
E               AttributeError: 'int' object has no attribute 'clear'

/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:766: AttributeError
_______________________________________________________ test_all_estimators[estimator4] _______________________________________________________

estimator = ('SerializableShapeletModel', <class 'tslearn.shapelets.SerializableShapeletModel'>)

    @pytest.mark.parametrize('estimator', get_estimators('all'))
    def test_all_estimators(estimator):
        """Test all the estimators in tslearn."""
        allow_nan = (hasattr(checks, 'ALLOW_NAN') and
                     _safe_tags(estimator[1](), "allow_nan"))
        if allow_nan:
            checks.ALLOW_NAN.append(estimator[0])
>       check_estimator(estimator[1])

test_estimators.py:178: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_estimators.py:148: in check_estimator
    check_parameters_default_constructible(name, Estimator)
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/estimator_checks.py:2146: in check_parameters_default_constructible
    with ignore_warnings(category=(DeprecationWarning, FutureWarning)):
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:375: in __enter__
    clean_warning_registry()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def clean_warning_registry():
        """Clean Python warning registry for easier testing of warning messages.
    
        We may not need to do this any more when getting rid of Python 2, not
        entirely sure. See https://bugs.python.org/issue4180 and
        https://bugs.python.org/issue21724 for more details.
    
        """
        reg = "__warningregistry__"
        for mod_name, mod in list(sys.modules.items()):
            if hasattr(mod, reg):
>               getattr(mod, reg).clear()
E               AttributeError: 'int' object has no attribute 'clear'

/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:766: AttributeError
_______________________________________________________ test_all_estimators[estimator5] _______________________________________________________

estimator = ('ShapeletModel', <class 'tslearn.shapelets.ShapeletModel'>)

    @pytest.mark.parametrize('estimator', get_estimators('all'))
    def test_all_estimators(estimator):
        """Test all the estimators in tslearn."""
        allow_nan = (hasattr(checks, 'ALLOW_NAN') and
                     _safe_tags(estimator[1](), "allow_nan"))
        if allow_nan:
            checks.ALLOW_NAN.append(estimator[0])
>       check_estimator(estimator[1])

test_estimators.py:178: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_estimators.py:148: in check_estimator
    check_parameters_default_constructible(name, Estimator)
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/estimator_checks.py:2146: in check_parameters_default_constructible
    with ignore_warnings(category=(DeprecationWarning, FutureWarning)):
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:375: in __enter__
    clean_warning_registry()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def clean_warning_registry():
        """Clean Python warning registry for easier testing of warning messages.
    
        We may not need to do this any more when getting rid of Python 2, not
        entirely sure. See https://bugs.python.org/issue4180 and
        https://bugs.python.org/issue21724 for more details.
    
        """
        reg = "__warningregistry__"
        for mod_name, mod in list(sys.modules.items()):
            if hasattr(mod, reg):
>               getattr(mod, reg).clear()
E               AttributeError: 'int' object has no attribute 'clear'

/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:766: AttributeError
_______________________________________________________ test_all_estimators[estimator6] _______________________________________________________

estimator = ('TimeSeriesKMeans', <class 'tslearn.clustering.TimeSeriesKMeans'>)

    @pytest.mark.parametrize('estimator', get_estimators('all'))
    def test_all_estimators(estimator):
        """Test all the estimators in tslearn."""
        allow_nan = (hasattr(checks, 'ALLOW_NAN') and
                     _safe_tags(estimator[1](), "allow_nan"))
        if allow_nan:
            checks.ALLOW_NAN.append(estimator[0])
>       check_estimator(estimator[1])

test_estimators.py:178: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_estimators.py:148: in check_estimator
    check_parameters_default_constructible(name, Estimator)
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/estimator_checks.py:2146: in check_parameters_default_constructible
    with ignore_warnings(category=(DeprecationWarning, FutureWarning)):
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:375: in __enter__
    clean_warning_registry()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def clean_warning_registry():
        """Clean Python warning registry for easier testing of warning messages.
    
        We may not need to do this any more when getting rid of Python 2, not
        entirely sure. See https://bugs.python.org/issue4180 and
        https://bugs.python.org/issue21724 for more details.
    
        """
        reg = "__warningregistry__"
        for mod_name, mod in list(sys.modules.items()):
            if hasattr(mod, reg):
>               getattr(mod, reg).clear()
E               AttributeError: 'int' object has no attribute 'clear'

/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:766: AttributeError
_______________________________________________________ test_all_estimators[estimator7] _______________________________________________________

estimator = ('TimeSeriesSVC', <class 'tslearn.svm.TimeSeriesSVC'>)

    @pytest.mark.parametrize('estimator', get_estimators('all'))
    def test_all_estimators(estimator):
        """Test all the estimators in tslearn."""
        allow_nan = (hasattr(checks, 'ALLOW_NAN') and
                     _safe_tags(estimator[1](), "allow_nan"))
        if allow_nan:
            checks.ALLOW_NAN.append(estimator[0])
>       check_estimator(estimator[1])

test_estimators.py:178: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_estimators.py:148: in check_estimator
    check_parameters_default_constructible(name, Estimator)
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/estimator_checks.py:2146: in check_parameters_default_constructible
    with ignore_warnings(category=(DeprecationWarning, FutureWarning)):
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:375: in __enter__
    clean_warning_registry()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def clean_warning_registry():
        """Clean Python warning registry for easier testing of warning messages.
    
        We may not need to do this any more when getting rid of Python 2, not
        entirely sure. See https://bugs.python.org/issue4180 and
        https://bugs.python.org/issue21724 for more details.
    
        """
        reg = "__warningregistry__"
        for mod_name, mod in list(sys.modules.items()):
            if hasattr(mod, reg):
>               getattr(mod, reg).clear()
E               AttributeError: 'int' object has no attribute 'clear'

/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:766: AttributeError
_______________________________________________________ test_all_estimators[estimator8] _______________________________________________________

estimator = ('TimeSeriesSVR', <class 'tslearn.svm.TimeSeriesSVR'>)

    @pytest.mark.parametrize('estimator', get_estimators('all'))
    def test_all_estimators(estimator):
        """Test all the estimators in tslearn."""
        allow_nan = (hasattr(checks, 'ALLOW_NAN') and
                     _safe_tags(estimator[1](), "allow_nan"))
        if allow_nan:
            checks.ALLOW_NAN.append(estimator[0])
>       check_estimator(estimator[1])

test_estimators.py:178: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
test_estimators.py:148: in check_estimator
    check_parameters_default_constructible(name, Estimator)
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/estimator_checks.py:2146: in check_parameters_default_constructible
    with ignore_warnings(category=(DeprecationWarning, FutureWarning)):
/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:375: in __enter__
    clean_warning_registry()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def clean_warning_registry():
        """Clean Python warning registry for easier testing of warning messages.
    
        We may not need to do this any more when getting rid of Python 2, not
        entirely sure. See https://bugs.python.org/issue4180 and
        https://bugs.python.org/issue21724 for more details.
    
        """
        reg = "__warningregistry__"
        for mod_name, mod in list(sys.modules.items()):
            if hasattr(mod, reg):
>               getattr(mod, reg).clear()
E               AttributeError: 'int' object has no attribute 'clear'

/tmp/tslearn-env/lib/python3.7/site-packages/sklearn/utils/testing.py:766: AttributeError
_______________________________________________________________ test_shapelets ________________________________________________________________

    def test_shapelets():
        n, sz, d = 15, 10, 2
        rng = np.random.RandomState(0)
        time_series = rng.randn(n, sz, d)
        y = rng.randint(2, size=n)
        clf = ShapeletModel(n_shapelets_per_size={2: 5},
                            max_iter=1,
                            verbose=0,
                            optimizer="sgd",
                            random_state=0)
        clf.fit(time_series, y)
        np.testing.assert_allclose(clf.shapelets_[0],
                                   np.array([[0.56373, 0.494684],
                                             [1.235707, 1.119235]]),
                                   atol=1e-2)
        np.testing.assert_allclose(clf.predict(time_series),
                                   np.array([0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0,
                                             1, 0]))
    
        from sklearn.model_selection import cross_validate
        cross_validate(clf, time_series, y, cv=2)
    
        model = ShapeletModel(n_shapelets_per_size={3: 2, 4: 1},
                              max_iter = 1)
        model.fit(time_series, y)
        for shp, shp_bis in zip(model.shapelets_,
                                model.shapelets_as_time_series_):
            np.testing.assert_allclose(shp,
>                                      to_time_series(shp_bis, remove_nans=True))
E           AssertionError: 
E           Not equal to tolerance rtol=1e-07, atol=0
E           
E           (shapes (4, 2), (0, 2) mismatch)
E            x: array([[-0.073456, -0.050393],
E                  [-0.031255, -0.020855],
E                  [ 0.055282,  0.036814],
E                  [ 0.027849, -0.002043]])
E            y: array([], shape=(0, 2), dtype=float64)

test_shapelets.py:38: AssertionError

those in scikit-learn's clean_warning_registry are probably irrelevent, but the shape mismatch in test_shapelets is concerning. Will try to check other python versions later today or tomorrow.

[rth]

Are there use cases where this would be safer than lighter wheels without dependencies included?

As far as I understood fat wheels with --skip-python will only include non-python dependencies (e.g. dependencies on C or C++ libraries).

More specifically, for instance on Linux the size of the wheel is mostly due to the /lib/ folder containing,

ls lib 
libasan.so        libatomic.so.1      libgomp.so        libitm.so.1      liblsan.so.0.0.0      libtsan.so        libubsan.so.1
libasan.so.5      libatomic.so.1.2.0  libgomp.so.1      libitm.so.1.0.0  libquadmath.so        libtsan.so.0      libubsan.so.1.0.0
libasan.so.5.0.0  libgcc_s.so         libgomp.so.1.0.0  liblsan.so       libquadmath.so.0      libtsan.so.0.0.0
libatomic.so      libgcc_s.so.1       libitm.so         liblsan.so.0     libquadmath.so.0.0.0  libubsan.so

the Python dependencies are not included in the wheel.

[rth]

error: /install_name_tool: [...] cycc.so is not a Mach-O file

Following discussion in conda-incubator/conda-press#38 it looks like one actually need to build it on the target platform, which would explain why it fails on Linux for you and on MacOS for me. Why Windows succeeds is still unclear.

[rtavenar]

Thanks for the PR! I just pushed a source dist to PyPI. If everything goes well, conda-forge should trigger soon, and we could try to use the artifacts then.

One thing I have in mind: at least for testing, we could use TravisCI (maybe pointing to a dedicated repo) to test the following steps:

• conda press
• wheel install
• wheel tests

on all platforms/OS combinations.

But then, I am unsure how we could upload the wheels to PyPI without publishing my PyPI credentials. Is there a way?

[rtavenar]

OK, so I have tried to set up something there.

The idea is to try what I wrote in the previous post, but I have some issues:

1. on OSX, it fails at installing python...
2. on Linux, the wheel seems to build, and install goes fine, but when I try to run pytest --pyargs tslearn, no tests are found.

If you have any idea why, @rth , I'd be happy to read it :)

[rth]

@rtavenar Sorry for the delay, will try to have a look at that and the paper later today.