Merge branch 'main' into add-zstd-android

Author: freakboy3742

.github/CODEOWNERS

@@ -26,7 +26,7 @@ Modules/Setup*                @erlend-aasland
 **/*context*                  @1st1
 **/*genobject*                @markshannon
 **/*hamt*                     @1st1
-**/*jit*                      @brandtbucher @savannahostrowski
+**/*jit*                      @brandtbucher @savannahostrowski @diegorusso
 Objects/set*                  @rhettinger
 Objects/dict*                 @methane @markshannon
 Objects/typevarobject.c       @JelleZijlstra

Doc/c-api/code.rst

@@ -211,6 +211,71 @@ bound into a function.
    .. versionadded:: 3.12
 
 
+.. _c_codeobject_flags:
+
+Code Object Flags
+-----------------
+
+Code objects contain a bit-field of flags, which can be retrieved as the
+:attr:`~codeobject.co_flags` Python attribute (for example using
+:c:func:`PyObject_GetAttrString`), and set using a *flags* argument to
+:c:func:`PyUnstable_Code_New` and similar functions.
+
+Flags whose names start with ``CO_FUTURE_`` correspond to features normally
+selectable by :ref:`future statements <future>`. These flags can be used in
+:c:member:`PyCompilerFlags.cf_flags`.
+Note that many ``CO_FUTURE_`` flags are mandatory in current versions of
+Python, and setting them has no effect.
+
+The following flags are available.
+For their meaning, see the linked documentation of their Python equivalents.
+
+
+.. list-table::
+   :widths: auto
+   :header-rows: 1
+
+   * * Flag
+     * Meaning
+   * * .. c:macro:: CO_OPTIMIZED
+     * :py:data:`inspect.CO_OPTIMIZED`
+   * * .. c:macro:: CO_NEWLOCALS
+     * :py:data:`inspect.CO_NEWLOCALS`
+   * * .. c:macro:: CO_VARARGS
+     * :py:data:`inspect.CO_VARARGS`
+   * * .. c:macro:: CO_VARKEYWORDS
+     * :py:data:`inspect.CO_VARKEYWORDS`
+   * * .. c:macro:: CO_NESTED
+     * :py:data:`inspect.CO_NESTED`
+   * * .. c:macro:: CO_GENERATOR
+     * :py:data:`inspect.CO_GENERATOR`
+   * * .. c:macro:: CO_COROUTINE
+     * :py:data:`inspect.CO_COROUTINE`
+   * * .. c:macro:: CO_ITERABLE_COROUTINE
+     * :py:data:`inspect.CO_ITERABLE_COROUTINE`
+   * * .. c:macro:: CO_ASYNC_GENERATOR
+     * :py:data:`inspect.CO_ASYNC_GENERATOR`
+   * * .. c:macro:: CO_HAS_DOCSTRING
+     * :py:data:`inspect.CO_HAS_DOCSTRING`
+   * * .. c:macro:: CO_METHOD
+     * :py:data:`inspect.CO_METHOD`
+
+   * * .. c:macro:: CO_FUTURE_DIVISION
+     * no effect (:py:data:`__future__.division`)
+   * * .. c:macro:: CO_FUTURE_ABSOLUTE_IMPORT
+     * no effect (:py:data:`__future__.absolute_import`)
+   * * .. c:macro:: CO_FUTURE_WITH_STATEMENT
+     * no effect (:py:data:`__future__.with_statement`)
+   * * .. c:macro:: CO_FUTURE_PRINT_FUNCTION
+     * no effect (:py:data:`__future__.print_function`)
+   * * .. c:macro:: CO_FUTURE_UNICODE_LITERALS
+     * no effect (:py:data:`__future__.unicode_literals`)
+   * * .. c:macro:: CO_FUTURE_GENERATOR_STOP
+     * no effect (:py:data:`__future__.generator_stop`)
+   * * .. c:macro:: CO_FUTURE_ANNOTATIONS
+     * :py:data:`__future__.annotations`
+
+
 Extra information
 -----------------
 

Doc/c-api/structures.rst

@@ -28,18 +28,52 @@ under :ref:`reference counting <countingrefs>`.
    object.  In a normal "release" build, it contains only the object's
    reference count and a pointer to the corresponding type object.
    Nothing is actually declared to be a :c:type:`PyObject`, but every pointer
-   to a Python object can be cast to a :c:expr:`PyObject*`.  Access to the
-   members must be done by using the macros :c:macro:`Py_REFCNT` and
-   :c:macro:`Py_TYPE`.
+   to a Python object can be cast to a :c:expr:`PyObject*`.
+
+   The members must not be accessed directly; instead use macros such as
+   :c:macro:`Py_REFCNT` and :c:macro:`Py_TYPE`.
+
+   .. c:member:: Py_ssize_t ob_refcnt
+
+      The object's reference count, as returned by :c:macro:`Py_REFCNT`.
+      Do not use this field directly; instead use functions and macros such as
+      :c:macro:`!Py_REFCNT`, :c:func:`Py_INCREF` and :c:func:`Py_DecRef`.
+
+      The field type may be different from ``Py_ssize_t``, depending on
+      build configuration and platform.
+
+   .. c:member:: PyTypeObject* ob_type
+
+      The object's type.
+      Do not use this field directly; use :c:macro:`Py_TYPE` and
+      :c:func:`Py_SET_TYPE` instead.
 
 
 .. c:type:: PyVarObject
 
-   This is an extension of :c:type:`PyObject` that adds the :c:member:`~PyVarObject.ob_size`
-   field.  This is only used for objects that have some notion of *length*.
-   This type does not often appear in the Python/C API.
-   Access to the members must be done by using the macros
-   :c:macro:`Py_REFCNT`, :c:macro:`Py_TYPE`, and :c:macro:`Py_SIZE`.
+   An extension of :c:type:`PyObject` that adds the
+   :c:member:`~PyVarObject.ob_size` field.
+   This is intended for objects that have some notion of *length*.
+
+   As with :c:type:`!PyObject`, the members must not be accessed directly;
+   instead use macros such as :c:macro:`Py_SIZE`, :c:macro:`Py_REFCNT` and
+   :c:macro:`Py_TYPE`.
+
+   .. c:member:: Py_ssize_t ob_size
+
+      A size field, whose contents should be considered an object's internal
+      implementation detail.
+
+      Do not use this field directly; use :c:macro:`Py_SIZE` instead.
+
+      Object creation functions such as :c:func:`PyObject_NewVar` will
+      generally set this field to the requested size (number of items).
+      After creation, arbitrary values can be stored in :c:member:`!ob_size`
+      using :c:macro:`Py_SET_SIZE`.
+
+      To get an object's publicly exposed length, as returned by
+      the Python function :py:func:`len`, use :c:func:`PyObject_Length`
+      instead.
 
 
 .. c:macro:: PyObject_HEAD
@@ -103,9 +137,8 @@ under :ref:`reference counting <countingrefs>`.
 
    Get the type of the Python object *o*.
 
-   Return a :term:`borrowed reference`.
-
-   Use the :c:func:`Py_SET_TYPE` function to set an object type.
+   The returned reference is :term:`borrowed <borrowed reference>` from *o*.
+   Do not release it with :c:func:`Py_DECREF` or similar.
 
    .. versionchanged:: 3.11
       :c:func:`Py_TYPE()` is changed to an inline static function.
@@ -122,16 +155,26 @@ under :ref:`reference counting <countingrefs>`.
 
 .. c:function:: void Py_SET_TYPE(PyObject *o, PyTypeObject *type)
 
-   Set the object *o* type to *type*.
+   Set the type of object *o* to *type*, without any checking or reference
+   counting.
+
+   This is a very low-level operation.
+   Consider instead setting the Python attribute :attr:`~object.__class__`
+   using :c:func:`PyObject_SetAttrString` or similar.
+
+   Note that assigning an incompatible type can lead to undefined behavior.
+
+   If *type* is a :ref:`heap type <heap-types>`, the caller must create a
+   new reference to it.
+   Similarly, if the old type of *o* is a heap type, the caller must release
+   a reference to that type.
 
    .. versionadded:: 3.9
 
 
 .. c:function:: Py_ssize_t Py_SIZE(PyVarObject *o)
 
-   Get the size of the Python object *o*.
-
-   Use the :c:func:`Py_SET_SIZE` function to set an object size.
+   Get the :c:member:`~PyVarObject.ob_size` field of *o*.
 
    .. versionchanged:: 3.11
       :c:func:`Py_SIZE()` is changed to an inline static function.
@@ -140,7 +183,7 @@ under :ref:`reference counting <countingrefs>`.
 
 .. c:function:: void Py_SET_SIZE(PyVarObject *o, Py_ssize_t size)
 
-   Set the object *o* size to *size*.
+   Set the :c:member:`~PyVarObject.ob_size` field of *o* to *size*.
 
    .. versionadded:: 3.9
 

Doc/c-api/typeobj.rst

@@ -492,9 +492,9 @@ metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that it
 type objects) *must* have the :c:member:`~PyVarObject.ob_size` field.
 
 
-.. c:member:: Py_ssize_t PyObject.ob_refcnt
+:c:member:`PyObject.ob_refcnt`
 
-   This is the type object's reference count, initialized to ``1`` by the
+   The type object's reference count is initialized to ``1`` by the
    ``PyObject_HEAD_INIT`` macro.  Note that for :ref:`statically allocated type
    objects <static-types>`, the type's instances (objects whose :c:member:`~PyObject.ob_type`
    points back to the type) do *not* count as references.  But for
@@ -506,7 +506,7 @@ type objects) *must* have the :c:member:`~PyVarObject.ob_size` field.
    This field is not inherited by subtypes.
 
 
-.. c:member:: PyTypeObject* PyObject.ob_type
+:c:member:`PyObject.ob_type`
 
    This is the type's type, in other words its metatype.  It is initialized by the
    argument to the ``PyObject_HEAD_INIT`` macro, and its value should normally be
@@ -532,14 +532,13 @@ type objects) *must* have the :c:member:`~PyVarObject.ob_size` field.
 PyVarObject Slots
 -----------------
 
-.. c:member:: Py_ssize_t PyVarObject.ob_size
+:c:member:`PyVarObject.ob_size`
 
    For :ref:`statically allocated type objects <static-types>`, this should be
    initialized to zero. For :ref:`dynamically allocated type objects
    <heap-types>`, this field has a special internal meaning.
 
-   This field should be accessed using the :c:func:`Py_SIZE()` and
-   :c:func:`Py_SET_SIZE()` macros.
+   This field should be accessed using the :c:func:`Py_SIZE()` macro.
 
    **Inheritance:**
 

Doc/c-api/veryhigh.rst

@@ -361,7 +361,7 @@ the same library that the Python runtime is using.
       :py:mod:`!ast` Python module, which exports these constants under
       the same names.
 
-   .. c:var:: int CO_FUTURE_DIVISION
-
-      This bit can be set in *flags* to cause division operator ``/`` to be
-      interpreted as "true division" according to :pep:`238`.
+   The "``PyCF``" flags above can be combined with "``CO_FUTURE``" flags such
+   as :c:macro:`CO_FUTURE_ANNOTATIONS` to enable features normally
+   selectable using :ref:`future statements <future>`.
+   See :ref:`c_codeobject_flags` for a complete list.

Doc/conf.py

@@ -635,13 +635,14 @@
     'image': '_static/og-image.png',
     'line_color': '#3776ab',
 }
-ogp_custom_meta_tags = [
-    '<meta name="theme-color" content="#3776ab">',
-]
-if 'create-social-cards' not in tags:  # noqa: F821
-    # Define a static preview image when not creating social cards
-    ogp_image = '_static/og-image.png'
-    ogp_custom_meta_tags += [
-        '<meta property="og:image:width" content="200">',
-        '<meta property="og:image:height" content="200">',
+if 'builder_html' in tags:  # noqa: F821
+    ogp_custom_meta_tags = [
+        '<meta name="theme-color" content="#3776ab">',
     ]
+    if 'create-social-cards' not in tags:  # noqa: F821
+        # Define a static preview image when not creating social cards
+        ogp_image = '_static/og-image.png'
+        ogp_custom_meta_tags += [
+            '<meta property="og:image:width" content="200">',
+            '<meta property="og:image:height" content="200">',
+        ]

Doc/glossary.rst

@@ -1280,6 +1280,16 @@ Glossary
       and ending with double underscores.  Special methods are documented in
       :ref:`specialnames`.
 
+   standard library
+      The collection of :term:`packages <package>`, :term:`modules <module>`
+      and :term:`extension modules <extension module>` distributed as a part
+      of the official Python interpreter package.  The exact membership of the
+      collection may vary based on platform, available system libraries, or
+      other criteria.  Documentation can be found at :ref:`library-index`.
+
+      See also :data:`sys.stdlib_module_names` for a list of all possible
+      standard library module names.
+
    statement
       A statement is part of a suite (a "block" of code).  A statement is either
       an :term:`expression` or one of several constructs with a keyword, such
@@ -1290,6 +1300,9 @@ Glossary
       issues such as incorrect types. See also :term:`type hints <type hint>`
       and the :mod:`typing` module.
 
+   stdlib
+      An abbreviation of :term:`standard library`.
+
    strong reference
       In Python's C API, a strong reference is a reference to an object
       which is owned by the code holding the reference.  The strong

Doc/howto/logging-cookbook.rst

@@ -4078,6 +4078,104 @@ lines. With this approach, you get better output:
     WARNING:demo:    1/0
     WARNING:demo:ZeroDivisionError: division by zero
 
+How to uniformly handle newlines in logging output
+--------------------------------------------------
+
+Usually, messages that are logged (say to console or file) consist of a single
+line of text. However, sometimes there is a need to handle messages with
+multiple lines - whether because a logging format string contains newlines, or
+logged data contains newlines. If you want to handle such messages uniformly, so
+that each line in the logged message appears uniformly formatted as if it was
+logged separately, you can do this using a handler mixin, as in the following
+snippet:
+
+.. code-block:: python
+
+    # Assume this is in a module mymixins.py
+    import copy
+
+    class MultilineMixin:
+        def emit(self, record):
+            s = record.getMessage()
+            if '\n' not in s:
+                super().emit(record)
+            else:
+                lines = s.splitlines()
+                rec = copy.copy(record)
+                rec.args = None
+                for line in lines:
+                    rec.msg = line
+                    super().emit(rec)
+
+You can use the mixin as in the following script:
+
+.. code-block:: python
+
+    import logging
+
+    from mymixins import MultilineMixin
+
+    logger = logging.getLogger(__name__)
+
+    class StreamHandler(MultilineMixin, logging.StreamHandler):
+        pass
+
+    if __name__ == '__main__':
+        logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(levelname)-9s %(message)s',
+                            handlers = [StreamHandler()])
+        logger.debug('Single line')
+        logger.debug('Multiple lines:\nfool me once ...')
+        logger.debug('Another single line')
+        logger.debug('Multiple lines:\n%s', 'fool me ...\ncan\'t get fooled again')
+
+The script, when run, prints something like:
+
+.. code-block:: text
+
+    2025-07-02 13:54:47,234 DEBUG     Single line
+    2025-07-02 13:54:47,234 DEBUG     Multiple lines:
+    2025-07-02 13:54:47,234 DEBUG     fool me once ...
+    2025-07-02 13:54:47,234 DEBUG     Another single line
+    2025-07-02 13:54:47,234 DEBUG     Multiple lines:
+    2025-07-02 13:54:47,234 DEBUG     fool me ...
+    2025-07-02 13:54:47,234 DEBUG     can't get fooled again
+
+If, on the other hand, you are concerned about `log injection
+<https://owasp.org/www-community/attacks/Log_Injection>`_, you can use a
+formatter which escapes newlines, as per the following example:
+
+.. code-block:: python
+
+    import logging
+
+    logger = logging.getLogger(__name__)
+
+    class EscapingFormatter(logging.Formatter):
+        def format(self, record):
+            s = super().format(record)
+            return s.replace('\n', r'\n')
+
+    if __name__ == '__main__':
+        h = logging.StreamHandler()
+        h.setFormatter(EscapingFormatter('%(asctime)s %(levelname)-9s %(message)s'))
+        logging.basicConfig(level=logging.DEBUG, handlers = [h])
+        logger.debug('Single line')
+        logger.debug('Multiple lines:\nfool me once ...')
+        logger.debug('Another single line')
+        logger.debug('Multiple lines:\n%s', 'fool me ...\ncan\'t get fooled again')
+
+You can, of course, use whatever escaping scheme makes the most sense for you.
+The script, when run, should produce output like this:
+
+.. code-block:: text
+
+    2025-07-09 06:47:33,783 DEBUG     Single line
+    2025-07-09 06:47:33,783 DEBUG     Multiple lines:\nfool me once ...
+    2025-07-09 06:47:33,783 DEBUG     Another single line
+    2025-07-09 06:47:33,783 DEBUG     Multiple lines:\nfool me ...\ncan't get fooled again
+
+Escaping behaviour can't be the stdlib default , as it would break backwards
+compatibility.
 
 .. patterns-to-avoid: