Merge remote-tracking branch 'upstream/main' into cpython-coverage.py

Author: swtaarrs

.github/CODEOWNERS

@@ -247,6 +247,7 @@ Lib/test/test_interpreters/   @ericsnowcurrently
 /Tools/wasm/                  @brettcannon
 
 # SBOM
+/Misc/externals.spdx.json     @sethmlarson
 /Misc/sbom.spdx.json          @sethmlarson
 /Tools/build/generate_sbom.py @sethmlarson
 

.github/workflows/jit.yml

@@ -70,13 +70,13 @@ jobs:
             runner: ubuntu-latest
             compiler: gcc
             # These fail because of emulation, not because of the JIT:
-            exclude: test_unix_events test_init test_process_pool test_shutdown test_multiprocessing_fork test_cmd_line test_faulthandler test_os test_perf_profiler test_posix test_signal test_socket test_subprocess test_threading test_venv
+            exclude: test_unix_events test_init test_process_pool test_shutdown test_multiprocessing_fork test_cmd_line test_faulthandler test_os test_perf_profiler test_posix test_signal test_socket test_subprocess test_threading test_venv test_external_inspection
           - target: aarch64-unknown-linux-gnu/clang
             architecture: aarch64
             runner: ubuntu-latest
             compiler: clang
             # These fail because of emulation, not because of the JIT:
-            exclude: test_unix_events test_init test_process_pool test_shutdown test_multiprocessing_fork test_cmd_line test_faulthandler test_os test_perf_profiler test_posix test_signal test_socket test_subprocess test_threading test_venv
+            exclude: test_unix_events test_init test_process_pool test_shutdown test_multiprocessing_fork test_cmd_line test_faulthandler test_os test_perf_profiler test_posix test_signal test_socket test_subprocess test_threading test_venv test_external_inspection
     env:
       CC: ${{ matrix.compiler }}
     steps:

Doc/library/abc.rst

@@ -101,11 +101,11 @@ a helper class :class:`ABC` to alternatively define ABCs through inheritance:
       subclass of the ABC.  (This class method is called from the
       :meth:`~class.__subclasscheck__` method of the ABC.)
 
-      This method should return ``True``, ``False`` or ``NotImplemented``.  If
+      This method should return ``True``, ``False`` or :data:`NotImplemented`.  If
       it returns ``True``, the *subclass* is considered a subclass of this ABC.
       If it returns ``False``, the *subclass* is not considered a subclass of
       this ABC, even if it would normally be one.  If it returns
-      ``NotImplemented``, the subclass check is continued with the usual
+      :data:`!NotImplemented`, the subclass check is continued with the usual
       mechanism.
 
       .. XXX explain the "usual mechanism"

Doc/library/ast.rst

@@ -2183,14 +2183,17 @@ and classes for traversing abstract syntax trees:
    modified to correspond to :pep:`484` "signature type comments",
    e.g. ``(str, int) -> List[str]``.
 
-   Also, setting ``feature_version`` to a tuple ``(major, minor)``
-   will attempt to parse using that Python version's grammar.
-   Currently ``major`` must equal to ``3``.  For example, setting
-   ``feature_version=(3, 4)`` will allow the use of ``async`` and
-   ``await`` as variable names.  The lowest supported version is
-   ``(3, 7)``; the highest is ``sys.version_info[0:2]``.
-
-   If source contains a null character ('\0'), :exc:`ValueError` is raised.
+   Setting ``feature_version`` to a tuple ``(major, minor)`` will result in
+   a "best-effort" attempt to parse using that Python version's grammar.
+   For example, setting ``feature_version=(3, 9)`` will attempt to disallow
+   parsing of :keyword:`match` statements.
+   Currently ``major`` must equal to ``3``. The lowest supported version is
+   ``(3, 7)`` (and this may increase in future Python versions);
+   the highest is ``sys.version_info[0:2]``. "Best-effort" attempt means there
+   is no guarantee that the parse (or success of the parse) is the same as
+   when run on the Python version corresponding to ``feature_version``.
+
+   If source contains a null character (``\0``), :exc:`ValueError` is raised.
 
    .. warning::
       Note that successfully parsing source code into an AST object doesn't

Doc/library/constants.rst

@@ -33,27 +33,27 @@ A small number of constants live in the built-in namespace.  They are:
    the other type; may be returned by the in-place binary special methods
    (e.g. :meth:`~object.__imul__`, :meth:`~object.__iand__`, etc.) for the same purpose.
    It should not be evaluated in a boolean context.
-   ``NotImplemented`` is the sole instance of the :data:`types.NotImplementedType` type.
+   :data:`!NotImplemented` is the sole instance of the :data:`types.NotImplementedType` type.
 
    .. note::
 
-      When a binary (or in-place) method returns ``NotImplemented`` the
+      When a binary (or in-place) method returns :data:`!NotImplemented` the
       interpreter will try the reflected operation on the other type (or some
       other fallback, depending on the operator).  If all attempts return
-      ``NotImplemented``, the interpreter will raise an appropriate exception.
-      Incorrectly returning ``NotImplemented`` will result in a misleading
-      error message or the ``NotImplemented`` value being returned to Python code.
+      :data:`!NotImplemented`, the interpreter will raise an appropriate exception.
+      Incorrectly returning :data:`!NotImplemented` will result in a misleading
+      error message or the :data:`!NotImplemented` value being returned to Python code.
 
       See :ref:`implementing-the-arithmetic-operations` for examples.
 
    .. note::
 
-      ``NotImplementedError`` and ``NotImplemented`` are not interchangeable,
+      ``NotImplementedError`` and :data:`!NotImplemented` are not interchangeable,
       even though they have similar names and purposes.
       See :exc:`NotImplementedError` for details on when to use it.
 
    .. versionchanged:: 3.9
-      Evaluating ``NotImplemented`` in a boolean context is deprecated. While
+      Evaluating :data:`!NotImplemented` in a boolean context is deprecated. While
       it currently evaluates as true, it will emit a :exc:`DeprecationWarning`.
       It will raise a :exc:`TypeError` in a future version of Python.
 

Doc/library/datetime.rst

@@ -1209,6 +1209,9 @@ Supported operations:
 
    Naive and aware :class:`!datetime` objects are never equal.
 
+   If both comparands are aware, and have the same :attr:`!tzinfo` attribute,
+   the :attr:`!tzinfo` and :attr:`~.datetime.fold` attributes are ignored and
+   the base datetimes are compared.
    If both comparands are aware and have different :attr:`~.datetime.tzinfo`
    attributes, the comparison acts as comparands were first converted to UTC
    datetimes except that the implementation never overflows.
@@ -1222,6 +1225,9 @@ Supported operations:
    Order comparison between naive and aware :class:`.datetime` objects
    raises :exc:`TypeError`.
 
+   If both comparands are aware, and have the same :attr:`!tzinfo` attribute,
+   the :attr:`!tzinfo` and :attr:`~.datetime.fold` attributes are ignored and
+   the base datetimes are compared.
    If both comparands are aware and have different :attr:`~.datetime.tzinfo`
    attributes, the comparison acts as comparands were first converted to UTC
    datetimes except that the implementation never overflows.
@@ -1778,8 +1784,8 @@ Naive and aware :class:`!time` objects are never equal.
 Order comparison between naive and aware :class:`!time` objects raises
 :exc:`TypeError`.
 
-If both comparands are aware, and have
-the same :attr:`~.time.tzinfo` attribute, the common :attr:`!tzinfo` attribute is
+If both comparands are aware, and have the same :attr:`~.time.tzinfo`
+attribute, the :attr:`!tzinfo` and :attr:`!fold` attributes are
 ignored and the base times are compared. If both comparands are aware and
 have different :attr:`!tzinfo` attributes, the comparands are first adjusted by
 subtracting their UTC offsets (obtained from ``self.utcoffset()``).

Doc/library/enum.rst

@@ -400,6 +400,9 @@ Data Types
 
       results in the call ``int('1a', 16)`` and a value of ``17`` for the member.
 
+      ..note:: When writing a custom ``__new__``, do not use ``super().__new__`` --
+               call the appropriate ``__new__`` instead.
+
    .. method:: Enum.__repr__(self)
 
       Returns the string used for *repr()* calls.  By default, returns the

Doc/library/exceptions.rst

@@ -335,9 +335,9 @@ The following exceptions are the exceptions that are usually raised.
 
    .. note::
 
-      ``NotImplementedError`` and ``NotImplemented`` are not interchangeable,
+      ``NotImplementedError`` and :data:`NotImplemented` are not interchangeable,
       even though they have similar names and purposes.  See
-      :data:`NotImplemented` for details on when to use it.
+      :data:`!NotImplemented` for details on when to use it.
 
 .. exception:: OSError([arg])
                OSError(errno, strerror[, filename[, winerror[, filename2]]])

Doc/library/http.server.rst

@@ -520,6 +520,12 @@ the ``--cgi`` option::
    :mod:`http.server` command line ``--cgi`` support is being removed
    because :class:`CGIHTTPRequestHandler` is being removed.
 
+.. warning::
+
+   :class:`CGIHTTPRequestHandler` and the ``--cgi`` command line option
+   are not intended for use by untrusted clients and may be vulnerable
+   to exploitation. Always use within a secure environment.
+
 .. _http.server-security:
 
 Security Considerations

Doc/library/importlib.rst

@@ -265,7 +265,7 @@ ABC hierarchy::
       when invalidating the caches of all finders on :data:`sys.meta_path`.
 
       .. versionchanged:: 3.4
-         Returns ``None`` when called instead of ``NotImplemented``.
+         Returns ``None`` when called instead of :data:`NotImplemented`.
 
 
 .. class:: PathEntryFinder

Doc/library/inspect.rst

@@ -665,9 +665,6 @@ function.
    Accepts a wide range of Python callables, from plain functions and classes to
    :func:`functools.partial` objects.
 
-   If the passed object has a ``__signature__`` attribute, this function
-   returns it without further computations.
-
    For objects defined in modules using stringized annotations
    (``from __future__ import annotations``), :func:`signature` will
    attempt to automatically un-stringize the annotations using
@@ -702,6 +699,13 @@ function.
       Python.  For example, in CPython, some built-in functions defined in
       C provide no metadata about their arguments.
 
+   .. impl-detail::
+
+      If the passed object has a :attr:`!__signature__` attribute,
+      we may use it to create the signature.
+      The exact semantics are an implementation detail and are subject to
+      unannounced changes. Consult the source code for current semantics.
+
 
 .. class:: Signature(parameters=None, *, return_annotation=Signature.empty)
 

Doc/library/itertools.rst

@@ -778,7 +778,7 @@ The primary purpose of the itertools recipes is educational.  The recipes show
 various ways of thinking about individual tools — for example, that
 ``chain.from_iterable`` is related to the concept of flattening.  The recipes
 also give ideas about ways that the tools can be combined — for example, how
-``compress()`` and ``range()`` can work together.  The recipes also show patterns
+``starmap()`` and ``repeat()`` can work together.  The recipes also show patterns
 for using itertools with the :mod:`operator` and :mod:`collections` modules as
 well as with the built-in itertools such as ``map()``, ``filter()``,
 ``reversed()``, and ``enumerate()``.
@@ -863,10 +863,9 @@ which incur interpreter overhead.
        "Given a predicate that returns True or False, count the True results."
        return sum(map(pred, iterable))
 
-   def all_equal(iterable):
+   def all_equal(iterable, key=None):
        "Returns True if all the elements are equal to each other."
-       g = groupby(iterable)
-       return next(g, True) and not next(g, False)
+       return len(take(2, groupby(iterable, key))) <= 1
 
    def first_true(iterable, default=False, pred=None):
        """Returns the first true value in the iterable.
@@ -984,36 +983,17 @@ which incur interpreter overhead.
        """ Call a function repeatedly until an exception is raised.
 
        Converts a call-until-exception interface to an iterator interface.
-       Like builtins.iter(func, sentinel) but uses an exception instead
-       of a sentinel to end the loop.
-
-       Priority queue iterator:
-           iter_except(functools.partial(heappop, h), IndexError)
-
-       Non-blocking dictionary iterator:
-           iter_except(d.popitem, KeyError)
-
-       Non-blocking deque iterator:
-           iter_except(d.popleft, IndexError)
-
-       Non-blocking iterator over a producer Queue:
-           iter_except(q.get_nowait, Queue.Empty)
-
-       Non-blocking set iterator:
-           iter_except(s.pop, KeyError)
-
        """
+       # iter_except(d.popitem, KeyError) --> non-blocking dictionary iterator
        try:
            if first is not None:
-               # For database APIs needing an initial call to db.first()
                yield first()
            while True:
                yield func()
        except exception:
            pass
 
 
-
 The following recipes have a more mathematical flavor:
 
 .. testcode::
@@ -1225,6 +1205,8 @@ The following recipes have a more mathematical flavor:
 
     >>> [all_equal(s) for s in ('', 'A', 'AAAA', 'AAAB', 'AAABA')]
     [True, True, True, False, False]
+    >>> [all_equal(s, key=str.casefold) for s in ('', 'A', 'AaAa', 'AAAB', 'AAABA')]
+    [True, True, True, False, False]
 
     >>> quantify(range(99), lambda x: x%2==0)
     50

Doc/library/logging.rst

@@ -77,6 +77,27 @@ is the module's name in the Python package namespace.
 
 .. class:: Logger
 
+   .. attribute:: Logger.name
+
+      This is the logger's name, and is the value that was passed to :func:`getLogger`
+      to obtain the logger.
+
+      .. note:: This attribute should be treated as read-only.
+
+   .. attribute:: Logger.level
+
+      The threshold of this logger, as set by the :meth:`setLevel` method.
+
+      .. note:: Do not set this attribute directly - always use :meth:`setLevel`,
+         which has checks for the level passed to it.
+
+   .. attribute:: Logger.parent
+
+      The parent logger of this logger. It may change based on later instantiation
+      of loggers which are higher up in the namespace hierarchy.
+
+      .. note:: This value should be treated as read-only.
+
    .. attribute:: Logger.propagate
 
       If this attribute evaluates to true, events logged to this logger will be
@@ -108,6 +129,21 @@ is the module's name in the Python package namespace.
          scenario is to attach handlers only to the root logger, and to let
          propagation take care of the rest.
 
+   .. attribute:: Logger.handlers
+
+      The list of handlers directly attached to this logger instance.
+
+      .. note:: This attribute should be treated as read-only; it is normally changed via
+         the :meth:`addHandler` and :meth:`removeHandler` methods, which use locks to ensure
+         thread-safe operation.
+
+   .. attribute:: Logger.disabled
+
+      This attribute disables handling of any events. It is set to ``False`` in the
+      initializer, and only changed by logging configuration code.
+
+      .. note:: This attribute should be treated as read-only.
+
    .. method:: Logger.setLevel(level)
 
       Sets the threshold for this logger to *level*. Logging messages which are less

Doc/library/numbers.rst

@@ -166,7 +166,7 @@ Complex``. I'll consider ``a + b``:
 2. If ``A`` falls back to the boilerplate code, and it were to
    return a value from :meth:`~object.__add__`, we'd miss the possibility
    that ``B`` defines a more intelligent :meth:`~object.__radd__`, so the
-   boilerplate should return :const:`NotImplemented` from
+   boilerplate should return :data:`NotImplemented` from
    :meth:`!__add__`. (Or ``A`` may not implement :meth:`!__add__` at
    all.)
 3. Then ``B``'s :meth:`~object.__radd__` gets a chance. If it accepts

Doc/library/pickle.rst

@@ -377,7 +377,7 @@ The :mod:`pickle` module exports three classes, :class:`Pickler`,
       Special reducer that can be defined in :class:`Pickler` subclasses. This
       method has priority over any reducer in the :attr:`dispatch_table`.  It
       should conform to the same interface as a :meth:`~object.__reduce__` method, and
-      can optionally return ``NotImplemented`` to fallback on
+      can optionally return :data:`NotImplemented` to fallback on
       :attr:`dispatch_table`-registered reducers to pickle ``obj``.
 
       For a detailed example, see :ref:`reducer_override`.
@@ -503,7 +503,7 @@ What can be pickled and unpickled?
 The following types can be pickled:
 
 * built-in constants (``None``, ``True``, ``False``, ``Ellipsis``, and
-  ``NotImplemented``);
+  :data:`NotImplemented`);
 
 * integers, floating-point numbers, complex numbers;
 
@@ -905,7 +905,7 @@ functions and classes.
 For those cases, it is possible to subclass from the :class:`Pickler` class and
 implement a :meth:`~Pickler.reducer_override` method. This method can return an
 arbitrary reduction tuple (see :meth:`~object.__reduce__`). It can alternatively return
-``NotImplemented`` to fallback to the traditional behavior.
+:data:`NotImplemented` to fallback to the traditional behavior.
 
 If both the :attr:`~Pickler.dispatch_table` and
 :meth:`~Pickler.reducer_override` are defined, then

Doc/library/pyexpat.rst

@@ -196,6 +196,37 @@ XMLParser Objects
    :exc:`ExpatError` to be raised with the :attr:`code` attribute set to
    ``errors.codes[errors.XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING]``.
 
+.. method:: xmlparser.SetReparseDeferralEnabled(enabled)
+
+   .. warning::
+
+      Calling ``SetReparseDeferralEnabled(False)`` has security implications,
+      as detailed below; please make sure to understand these consequences
+      prior to using the ``SetReparseDeferralEnabled`` method.
+
+   Expat 2.6.0 introduced a security mechanism called "reparse deferral"
+   where instead of causing denial of service through quadratic runtime
+   from reparsing large tokens, reparsing of unfinished tokens is now delayed
+   by default until a sufficient amount of input is reached.
+   Due to this delay, registered handlers may — depending of the sizing of
+   input chunks pushed to Expat — no longer be called right after pushing new
+   input to the parser.  Where immediate feedback and taking over responsiblity
+   of protecting against denial of service from large tokens are both wanted,
+   calling ``SetReparseDeferralEnabled(False)`` disables reparse deferral
+   for the current Expat parser instance, temporarily or altogether.
+   Calling ``SetReparseDeferralEnabled(True)`` allows re-enabling reparse
+   deferral.
+
+   .. versionadded:: 3.13
+
+.. method:: xmlparser.GetReparseDeferralEnabled()
+
+   Returns whether reparse deferral is currently enabled for the given
+   Expat parser instance.
+
+   .. versionadded:: 3.13
+
+
 :class:`xmlparser` objects have the following attributes: