Filter out some distutils deprecation warnings with python 3.10

[tornaria]

As reported in #30766 comment:111

CC: @antonio-rojas

Component: misc

Author: Gonzalo Tornaría

Branch/Commit: 79234bb

Reviewer: Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/33135

[tornaria]

Branch: u/tornaria/deprecation_distutils

[tornaria]

New commits:

836249c

trac 33135: add filter to ignore deprecation warnings

[tornaria]

Author: Gonzalo Tornaría

[tornaria]

Commit: 836249c

[kiwifb]

comment:2

I see the warning a lot during docbuilding coming from matplotlib (building on sage-on-gentoo with python 3.10 right now)

[quadratic] /usr/lib/python3.10/site-packages/matplotlib/__init__.py:88: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives

for example. Should it be added to the warnings to catch?

[tornaria]

comment:3

I wonder if filters should strive be more specific (which would be itself a more precise form of documenting the filter). For example, the pythran warning in the context of this patch could have

message='`np.(bool|float|complex)` is a deprecated alias'

which catches the deprecation warnings and is explicit about what is being ignored.

---

WRT the current patch, there are two uses of distutils in sage source code:

• distutils.sysconfig.get_python_inc() is deprecated, it has no replacement in sysconfig
• distutils.errors.CCompilerError is not present in setuptools.errors

So I have no clue how to remove these two uses of distutil.

The third one is in numpy, not yet fixed upstream (see numpy/numpy#20225)

[tornaria]

comment:4

Replying to @kiwifb:

I see the warning a lot during docbuilding coming from matplotlib (building on sage-on-gentoo with python 3.10 right now)

[quadratic] /usr/lib/python3.10/site-packages/matplotlib/__init__.py:88: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives

for example. Should it be added to the warnings to catch?

I don't get that one. Are you using matplotlib 3.5?

[kiwifb]

comment:5

Right, I am still at 3.4.3 in Gentoo (on the stable channel), switching to 3.5 is easy if that fixes it.

[mkoeppe]

Reviewer: Matthias Koeppe

[kiwifb]

comment:7

For distutils.sysconfig.get_python_inc() we could use sysconfig.get_config_var('INCLUDEPY') which should give the same value.

[antonio-rojas]

comment:8

I'm still getting lots of warnings coming from cython

sage -t --long --random-seed=173302895909105717773601440401632228334 /usr/lib/python3.10/site-packages/sage/env.py
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage/env.py", line 440, in sage.env.?
Failed example:
    cython('''
    #distutils: extra_compile_args = OPENMP_CFLAGS
    #distutils: extra_link_args = OPENMP_CFLAGS
    from cython.parallel import prange

    cdef int i
    cdef int n = 30
    cdef int sum = 0

    for i in prange(n, num_threads=4, nogil=True):
        sum += i

    print(sum)
    ''')
Expected:
    435
Got:
    doctest:warning
      File "/usr/bin/sage-runtests", line 155, in <module>
        err = DC.run()
      File "/usr/lib/python3.10/site-packages/sage/doctest/control.py", line 1256, in run
        self.run_doctests()
      File "/usr/lib/python3.10/site-packages/sage/doctest/control.py", line 957, in run_doctests
        self.dispatcher.dispatch()
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 2004, in dispatch
        self.parallel_dispatch()
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 1899, in parallel_dispatch
        w.start()  # This might take some time
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 2173, in start
        super(DocTestWorker, self).start()
      File "/usr/lib/python3.10/multiprocessing/process.py", line 121, in start
        self._popen = self._Popen(self)
      File "/usr/lib/python3.10/multiprocessing/context.py", line 224, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/usr/lib/python3.10/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
        self._launch(process_obj)
      File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 71, in _launch
        code = process_obj._bootstrap(parent_sentinel=child_r)
      File "/usr/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
        self.run()
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 2145, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 2475, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 2527, in _run
        result = runner.run(test)
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 861, in run
        return self._run(test, compileflags, out)
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 694, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.env.?[5]>", line 1, in <module>
        cython('''
      File "/usr/lib/python3.10/site-packages/sage/misc/cython.py", line 661, in cython_compile
        return cython_import_all(tmpfile, get_globals(), **kwds)
      File "/usr/lib/python3.10/site-packages/sage/misc/cython.py", line 551, in cython_import_all
        m = cython_import(filename, **kwds)
      File "/usr/lib/python3.10/site-packages/sage/misc/cython.py", line 526, in cython_import
        name, build_dir = cython(filename, **kwds)
      File "/usr/lib/python3.10/site-packages/sage/misc/cython.py", line 288, in cython
        from Cython.Build import cythonize
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/usr/lib/python3.10/site-packages/Cython/Build/__init__.py", line 1, in <module>
        from .Dependencies import cythonize
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/usr/lib/python3.10/site-packages/Cython/Build/Dependencies.py", line 17, in <module>
        from distutils.extension import Extension
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/usr/lib/python3.10/distutils/__init__.py", line 19, in <module>
        warnings.warn(_DEPRECATION_MESSAGE,
      File "/usr/lib/python3.10/warnings.py", line 109, in _showwarnmsg
        sw(msg.message, msg.category, msg.filename, msg.lineno,
    :
    DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    doctest:warning
      File "/usr/bin/sage-runtests", line 155, in <module>
        err = DC.run()
      File "/usr/lib/python3.10/site-packages/sage/doctest/control.py", line 1256, in run
        self.run_doctests()
      File "/usr/lib/python3.10/site-packages/sage/doctest/control.py", line 957, in run_doctests
        self.dispatcher.dispatch()
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 2004, in dispatch
        self.parallel_dispatch()
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 1899, in parallel_dispatch
        w.start()  # This might take some time
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 2173, in start
        super(DocTestWorker, self).start()
      File "/usr/lib/python3.10/multiprocessing/process.py", line 121, in start
        self._popen = self._Popen(self)
      File "/usr/lib/python3.10/multiprocessing/context.py", line 224, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/usr/lib/python3.10/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
        self._launch(process_obj)
      File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 71, in _launch
        code = process_obj._bootstrap(parent_sentinel=child_r)
      File "/usr/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
        self.run()
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 2145, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 2475, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 2527, in _run
        result = runner.run(test)
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 861, in run
        return self._run(test, compileflags, out)
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 694, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.env.?[5]>", line 1, in <module>
        cython('''
      File "/usr/lib/python3.10/site-packages/sage/misc/cython.py", line 661, in cython_compile
        return cython_import_all(tmpfile, get_globals(), **kwds)
      File "/usr/lib/python3.10/site-packages/sage/misc/cython.py", line 551, in cython_import_all
        m = cython_import(filename, **kwds)
      File "/usr/lib/python3.10/site-packages/sage/misc/cython.py", line 526, in cython_import
        name, build_dir = cython(filename, **kwds)
      File "/usr/lib/python3.10/site-packages/sage/misc/cython.py", line 288, in cython
        from Cython.Build import cythonize
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/usr/lib/python3.10/site-packages/Cython/Build/__init__.py", line 2, in <module>
        from .Distutils import build_ext
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/usr/lib/python3.10/site-packages/Cython/Build/Distutils.py", line 1, in <module>
        from Cython.Distutils.build_ext import build_ext
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/usr/lib/python3.10/site-packages/Cython/Distutils/__init__.py", line 1, in <module>
        from Cython.Distutils.build_ext import build_ext
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/usr/lib/python3.10/site-packages/Cython/Distutils/build_ext.py", line 11, in <module>
        from distutils.command.build_ext import build_ext as _build_ext
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/usr/lib/python3.10/distutils/command/build_ext.py", line 13, in <module>
        from distutils.sysconfig import customize_compiler, get_python_version
      File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 883, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/usr/lib/python3.10/distutils/sysconfig.py", line 58, in <module>
        warnings.warn(
      File "/usr/lib/python3.10/warnings.py", line 109, in _showwarnmsg
        sw(msg.message, msg.category, msg.filename, msg.lineno,
    :
    DeprecationWarning: The distutils.sysconfig module is deprecated, use sysconfig instead
    435
**********************************************************************