gh-91246: Implement PEP 775: Make zlib required on platforms other than wasi

Doc/library/shutil.rst

@@ -601,8 +601,7 @@ provided.  They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
    any format-specific extension.
 
    *format* is the archive format: one of
-   "zip" (if the :mod:`zlib` module is available), "tar", "gztar" (if the
-   :mod:`zlib` module is available), "bztar" (if the :mod:`bz2` module is
+   "zip", "tar", "gztar", "bztar" (if the :mod:`bz2` module is
    available), or "xztar" (if the :mod:`lzma` module is available).
 
    *root_dir* is a directory that will be the root directory of the
@@ -653,9 +652,9 @@ provided.  They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
 
    By default :mod:`shutil` provides these formats:
 
-   - *zip*: ZIP file (if the :mod:`zlib` module is available).
+   - *zip*: ZIP file.
    - *tar*: Uncompressed tar file. Uses POSIX.1-2001 pax format for new archives.
-   - *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
+   - *gztar*: gzip'ed tar-file.
    - *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
    - *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
 
@@ -768,10 +767,9 @@ provided.  They rely on the :mod:`zipfile` and :mod:`tarfile` modules.
 
    By default :mod:`shutil` provides these formats:
 
-   - *zip*: ZIP file (unpacking compressed files works only if the corresponding
-     module is available).
+   - *zip*: ZIP file.
    - *tar*: uncompressed tar file.
-   - *gztar*: gzip'ed tar-file (if the :mod:`zlib` module is available).
+   - *gztar*: gzip'ed tar-file.
    - *bztar*: bzip2'ed tar-file (if the :mod:`bz2` module is available).
    - *xztar*: xz'ed tar-file (if the :mod:`lzma` module is available).
 

Doc/library/test.rst

@@ -748,7 +748,12 @@ The :mod:`test.support` module defines the following functions:
 
 .. decorator:: requires_zlib
 
-   Decorator for skipping tests if :mod:`zlib` doesn't exist.
+   Decorator for skipping tests if running on WASI as :mod:`zlib` is
+   unsupported.
+
+.. versionchanged:: next
+   Test is skipped only if it is running on WASI as builds without :mod:`zlib` on
+   other systems are unsupported.
 
 
 .. decorator:: requires_gzip

Doc/library/zlib.rst

@@ -9,14 +9,11 @@
 
 For applications that require data compression, the functions in this module
 allow compression and decompression, using the zlib library. The zlib library
-has its own home page at https://www.zlib.net.   There are known
-incompatibilities between the Python module and versions of the zlib library
-earlier than 1.1.3; 1.1.3 has a `security vulnerability <https://zlib.net/zlib_faq.html#faq33>`_, so we recommend using
-1.1.4 or later.
+has its own `home page <https://www.zlib.net>`_.
 
 zlib's functions have many options and often need to be used in a particular
 order.  This documentation doesn't attempt to cover all of the permutations;
-consult the zlib manual at http://www.zlib.net/manual.html for authoritative
+consult the `zlib manual <http://www.zlib.net/manual.html>`_ for authoritative
 information.
 
 For reading and writing ``.gz`` files see the :mod:`gzip` module.

Doc/using/configure.rst

@@ -32,6 +32,10 @@ Features and minimum versions required to build CPython:
 * Autoconf 2.72 and aclocal 1.16.5 are required to regenerate the
   :file:`configure` script.
 
+* `zlib <https://zlib.net>`_ 1.1.3 is the minium version and 1.1.4 is recommended due to
+   a `security vulnerability <https://zlib.net/zlib_faq.html#faq33>`_ for the :mod:`zlib`
+   extension module.
+
 .. versionchanged:: 3.1
    Tcl/Tk version 8.3.1 is now required.
 
@@ -61,6 +65,9 @@ Features and minimum versions required to build CPython:
 .. versionchanged:: 3.14
    Autoconf 2.72 is now required.
 
+.. versionchanged:: next
+   zlib 1.1.3 is now required.
+
 See also :pep:`7` "Style Guide for C Code" and :pep:`11` "CPython platform
 support".
 
@@ -454,7 +461,7 @@ Options for third-party dependencies
 .. option:: ZLIB_CFLAGS
 .. option:: ZLIB_LIBS
 
-   C compiler and linker flags for ``libzlib``, used by :mod:`gzip` module,
+   C compiler and linker flags for ``libzlib``, used by the :mod:`zlib` module,
    overriding ``pkg-config``.
 
 

Doc/whatsnew/3.14.rst

@@ -1310,6 +1310,10 @@ Build changes
 * GNU Autoconf 2.72 is now required to generate :file:`configure`.
   (Contributed by Erlend Aasland in :gh:`115765`.)
 
+* zlib is now required to build CPytho on all platforms except WASI.
+  For more information see :pep:`775`.
+  (Contributed by Stan Ulbrych and Gregory P. Smith in :gh:`91246`.)
+
 .. _whatsnew314-pep761:
 
 PEP 761: Discontinuation of PGP signatures

Lib/encodings/zlib_codec.py

@@ -6,7 +6,7 @@
 """
 
 import codecs
-import zlib # this codec needs the optional zlib module !
+import zlib
 
 ### Codec APIs
 

Lib/shutil.py

@@ -11,12 +11,6 @@
 import collections
 import errno
 
-try:
-    import zlib
-    del zlib
-    _ZLIB_SUPPORTED = True
-except ImportError:
-    _ZLIB_SUPPORTED = False
 
 try:
     import bz2
@@ -1000,7 +994,7 @@ def _make_tarball(base_name, base_dir, compress="gzip", verbose=0, dry_run=0,
     """
     if compress is None:
         tar_compression = ''
-    elif _ZLIB_SUPPORTED and compress == 'gzip':
+    elif compress == 'gzip':
         tar_compression = 'gz'
     elif _BZ2_SUPPORTED and compress == 'bzip2':
         tar_compression = 'bz2'
@@ -1119,13 +1113,11 @@ def _make_zipfile(base_name, base_dir, verbose=0, dry_run=0,
 _ARCHIVE_FORMATS = {
     'tar':   (_make_tarball, [('compress', None)],
               "uncompressed tar file"),
+    'gztar': (_make_tarball, [('compress', 'gzip')],
+              "gzip'ed tar-file"),
+    'zip': (_make_zipfile, [], "ZIP file")
 }
 
-if _ZLIB_SUPPORTED:
-    _ARCHIVE_FORMATS['gztar'] = (_make_tarball, [('compress', 'gzip')],
-                                "gzip'ed tar-file")
-    _ARCHIVE_FORMATS['zip'] = (_make_zipfile, [], "ZIP file")
-
 if _BZ2_SUPPORTED:
     _ARCHIVE_FORMATS['bztar'] = (_make_tarball, [('compress', 'bzip2')],
                                 "bzip2'ed tar-file")
@@ -1345,12 +1337,9 @@ def _unpack_tarfile(filename, extract_dir, *, filter=None):
 _UNPACK_FORMATS = {
     'tar':   (['.tar'], _unpack_tarfile, [], "uncompressed tar file"),
     'zip':   (['.zip'], _unpack_zipfile, [], "ZIP file"),
+    'gztar': (['.tar.gz', '.tgz'], _unpack_tarfile, [], "gzip'ed tar-file"),
 }
 
-if _ZLIB_SUPPORTED:
-    _UNPACK_FORMATS['gztar'] = (['.tar.gz', '.tgz'], _unpack_tarfile, [],
-                                "gzip'ed tar-file")
-
 if _BZ2_SUPPORTED:
     _UNPACK_FORMATS['bztar'] = (['.tar.bz2', '.tbz2'], _unpack_tarfile, [],
                                 "bzip2'ed tar-file")

Lib/tarfile.py

@@ -364,10 +364,7 @@ def __init__(self, name, mode, comptype, fileobj, bufsize,
 
         try:
             if comptype == "gz":
-                try:
-                    import zlib
-                except ImportError:
-                    raise CompressionError("zlib module is not available") from None
+                import zlib
                 self.zlib = zlib
                 self.crc = zlib.crc32(b"")
                 if mode == "r":

Lib/test/support/__init__.py

@@ -493,12 +493,8 @@ def dec(*args, **kwargs):
     float.__getformat__("double").startswith("IEEE"),
     "test requires IEEE 754 doubles")
 
-def requires_zlib(reason='requires zlib'):
-    try:
-        import zlib
-    except ImportError:
-        zlib = None
-    return unittest.skipUnless(zlib, reason)
+def requires_zlib(reason='zlib is unsupported on WASI'):
+    return unittest.skipIf(sys.platform == "wasi", reason)
 
 def requires_gzip(reason='requires gzip'):
     try:

Lib/test/test_zlib.py

@@ -1,15 +1,27 @@
 import unittest
 from test import support
-from test.support import import_helper
 import binascii
 import copy
 import pickle
 import random
 import sys
+import importlib
 from test.support import bigmemtest, _1G, _4G, is_s390x
 
-
-zlib = import_helper.import_module('zlib')
+# Building CPython without zlib is not supported except WASI.
+#
+# Anyone who wants build CPython this way should be prepared to patch it,
+# but the core team may help getting those patches to the main branch
+# (as that’s the place where multiple third parties can cooperate).
+#
+# For tests to pass without zlib, this file needs to be removed.
+
+try:
+    zlib = importlib.import_module('zlib')
+except ImportError as msg:
+    if sys.platform.startswith('wasi'):
+        raise unittest.SkipTest(str(msg))
+    raise ImportError("Building CPython without zlib is not supported")
 
 requires_Compress_copy = unittest.skipUnless(
         hasattr(zlib.compressobj(), "copy"),

Lib/zipfile/__init__.py

@@ -15,7 +15,7 @@
 import time
 
 try:
-    import zlib # We may need its compression method
+    import zlib
     crc32 = zlib.crc32
 except ImportError:
     zlib = None

Lib/zipimport.py

@@ -604,9 +604,7 @@ def _read_directory(archive):
 
 _importing_zlib = False
 
-# Return the zlib.decompress function object, or NULL if zlib couldn't
-# be imported. The function is cached when found, so subsequent calls
-# don't import zlib again.
+# Return the zlib.decompress function object or raise an import error.
 def _get_decompress_func():
     global _importing_zlib
     if _importing_zlib:
@@ -618,9 +616,6 @@ def _get_decompress_func():
     _importing_zlib = True
     try:
         from zlib import decompress
-    except Exception:
-        _bootstrap._verbose_message('zipimport: zlib UNAVAILABLE')
-        raise ZipImportError("can't decompress data; zlib not available")
     finally:
         _importing_zlib = False
 

Misc/NEWS.d/next/Build/2025-02-19-00-00-00.gh-issue-91246.ahsgd3.rst

@@ -0,0 +1,2 @@
+Make zlib required to build CPython with the exception of WASI implementing :pep:`775`.
+(Contributed by Stan Ulbrych and Gregory P. Smith in :gh:`130297`.)

Modules/binascii.c

@@ -60,7 +60,7 @@
 #include "Python.h"
 #include "pycore_long.h"          // _PyLong_DigitValue
 #include "pycore_strhex.h"        // _Py_strhex_bytes_with_sep()
-#ifdef USE_ZLIB_CRC32
+#ifndef NO_ZLIB_CRC32
 #  include "zlib.h"
 #endif
 
@@ -616,7 +616,7 @@ binascii_crc_hqx_impl(PyObject *module, Py_buffer *data, unsigned int crc)
     return PyLong_FromUnsignedLong(crc);
 }
 
-#ifndef USE_ZLIB_CRC32
+#ifdef NO_ZLIB_CRC32
 /*  Crc - 32 BIT ANSI X3.66 CRC checksum files
     Also known as: ISO 3307
 **********************************************************************|
@@ -749,7 +749,7 @@ internal_crc32(const unsigned char *bin_data, Py_ssize_t len, unsigned int crc)
     result = (crc ^ 0xFFFFFFFF);
     return result & 0xffffffff;
 }
-#endif  /* USE_ZLIB_CRC32 */
+#endif  /* NO_ZLIB_CRC32 */
 
 /*[clinic input]
 binascii.crc32 -> unsigned_int
@@ -765,9 +765,11 @@ static unsigned int
 binascii_crc32_impl(PyObject *module, Py_buffer *data, unsigned int crc)
 /*[clinic end generated code: output=52cf59056a78593b input=bbe340bc99d25aa8]*/
 
-#ifdef USE_ZLIB_CRC32
+#ifndef NO_ZLIB_CRC32
 /* This is the same as zlibmodule.c zlib_crc32_impl. It exists in two
- * modules for historical reasons. */
+ * modules for historical reasons. They should be consolidated in the future
+ * once WASI supports zlib.
+ */
 {
     /* Releasing the GIL for very small buffers is inefficient
        and may lower performance */
@@ -798,7 +800,7 @@ binascii_crc32_impl(PyObject *module, Py_buffer *data, unsigned int crc)
     }
     return crc & 0xffffffff;
 }
-#else  /* USE_ZLIB_CRC32 */
+#else  /* NO_ZLIB_CRC32 */
 {
     const unsigned char *bin_data = data->buf;
     Py_ssize_t len = data->len;
@@ -815,7 +817,7 @@ binascii_crc32_impl(PyObject *module, Py_buffer *data, unsigned int crc)
         return internal_crc32(bin_data, len, crc);
     }
 }
-#endif  /* USE_ZLIB_CRC32 */
+#endif  /* NO_ZLIB_CRC32 */
 
 /*[clinic input]
 binascii.b2a_hex

PCbuild/pythoncore.vcxproj

@@ -457,9 +457,7 @@
     <ClCompile Include="..\Modules\_weakref.c" />
     <ClCompile Include="..\Modules\arraymodule.c" />
     <ClCompile Include="..\Modules\atexitmodule.c" />
-    <ClCompile Include="..\Modules\binascii.c">
-      <PreprocessorDefinitions>USE_ZLIB_CRC32;%(PreprocessorDefinitions)</PreprocessorDefinitions>
-    </ClCompile>
+    <ClCompile Include="..\Modules\binascii.c" />
     <ClCompile Include="..\Modules\cmathmodule.c" />
     <ClCompile Include="..\Modules\_datetimemodule.c" />
     <ClCompile Include="..\Modules\errnomodule.c" />