Merge branch 'main' into issue-46771

Author: asvetlov

.github/workflows/doc.yml

@@ -48,7 +48,7 @@ jobs:
     - name: 'Build HTML documentation'
       run: make -C Doc/ PYTHON=../python SPHINXOPTS="-q -W --keep-going -j4" html
     - name: 'Upload'
-      uses: actions/upload-artifact@v2.2.4
+      uses: actions/upload-artifact@v2.3.1
       with:
         name: doc-html
         path: Doc/build/html

Doc/c-api/exceptions.rst

@@ -253,6 +253,14 @@ For convenience, some of these functions will always return a
    .. versionadded:: 3.3
 
 
+.. c:function:: PyObject* PyErr_SetImportErrorSubclass(PyObject *exception, PyObject *msg, PyObject *name, PyObject *path)
+
+   Much like :c:func:`PyErr_SetImportError` but this function allows for
+   specifying a subclass of :exc:`ImportError` to raise.
+
+   .. versionadded:: 3.6
+
+
 .. c:function:: void PyErr_SyntaxLocationObject(PyObject *filename, int lineno, int col_offset)
 
    Set file, line, and offset information for the current exception.  If the
@@ -320,13 +328,6 @@ an error value).
    :mod:`warnings` module and the :option:`-W` option in the command line
    documentation.  There is no C API for warning control.
 
-.. c:function:: PyObject* PyErr_SetImportErrorSubclass(PyObject *exception, PyObject *msg, PyObject *name, PyObject *path)
-
-   Much like :c:func:`PyErr_SetImportError` but this function allows for
-   specifying a subclass of :exc:`ImportError` to raise.
-
-   .. versionadded:: 3.6
-
 
 .. c:function:: int PyErr_WarnExplicitObject(PyObject *category, PyObject *message, PyObject *filename, int lineno, PyObject *module, PyObject *registry)
 

Doc/c-api/reflection.rst

@@ -31,7 +31,7 @@ Reflection
    See also :c:func:`PyThreadState_GetFrame`.
 
 
-.. c:function:: int PyFrame_GetBack(PyFrameObject *frame)
+.. c:function:: PyFrameObject* PyFrame_GetBack(PyFrameObject *frame)
 
    Get the *frame* next outer frame.
 
@@ -42,7 +42,7 @@ Reflection
    .. versionadded:: 3.9
 
 
-.. c:function:: int PyFrame_GetCode(PyFrameObject *frame)
+.. c:function:: PyCodeObject* PyFrame_GetCode(PyFrameObject *frame)
 
    Get the *frame* code.
 

Doc/c-api/typeobj.rst

@@ -476,7 +476,7 @@ PyObject Slots
 --------------
 
 The type object structure extends the :c:type:`PyVarObject` structure. The
-:attr:`ob_size` field is used for dynamic types (created by  :func:`type_new`,
+:attr:`ob_size` field is used for dynamic types (created by :func:`type_new`,
 usually called from a class statement). Note that :c:data:`PyType_Type` (the
 metatype) initializes :c:member:`~PyTypeObject.tp_itemsize`, which means that its instances (i.e.
 type objects) *must* have the :attr:`ob_size` field.
@@ -2000,6 +2000,17 @@ and :c:type:`PyType_Type` effectively act as defaults.)
    For this field to be taken into account (even through inheritance),
    you must also set the :const:`Py_TPFLAGS_HAVE_FINALIZE` flags bit.
 
+   Also, note that, in a garbage collected Python,
+   :c:member:`~PyTypeObject.tp_dealloc` may be called from
+   any Python thread, not just the thread which created the object (if the object
+   becomes part of a refcount cycle, that cycle might be collected by a garbage
+   collection on any thread).  This is not a problem for Python API calls, since
+   the thread on which tp_dealloc is called will own the Global Interpreter Lock
+   (GIL). However, if the object being destroyed in turn destroys objects from some
+   other C or C++ library, care should be taken to ensure that destroying those
+   objects on the thread which called tp_dealloc will not violate any assumptions
+   of the library.
+
    **Inheritance:**
 
    This field is inherited by subtypes.
@@ -2024,17 +2035,6 @@ and :c:type:`PyType_Type` effectively act as defaults.)
    .. versionadded:: 3.9 (the field exists since 3.8 but it's only used since 3.9)
 
 
-Also, note that, in a garbage collected Python, :c:member:`~PyTypeObject.tp_dealloc` may be called from
-any Python thread, not just the thread which created the object (if the object
-becomes part of a refcount cycle, that cycle might be collected by a garbage
-collection on any thread).  This is not a problem for Python API calls, since
-the thread on which tp_dealloc is called will own the Global Interpreter Lock
-(GIL). However, if the object being destroyed in turn destroys objects from some
-other C or C++ library, care should be taken to ensure that destroying those
-objects on the thread which called tp_dealloc will not violate any assumptions
-of the library.
-
-
 .. _static-types:
 
 Static Types
@@ -2440,7 +2440,8 @@ Async Object Structures
 
       PyObject *am_aiter(PyObject *self);
 
-   Must return an :term:`awaitable` object.  See :meth:`__anext__` for details.
+   Must return an :term:`asynchronous iterator` object.
+   See :meth:`__anext__` for details.
 
    This slot may be set to ``NULL`` if an object does not implement
    asynchronous iteration protocol.

Doc/c-api/unicode.rst

@@ -1003,7 +1003,7 @@ Error handling is set by errors which may also be set to ``NULL`` meaning to use
 the default handling defined for the codec.  Default error handling for all
 built-in codecs is "strict" (:exc:`ValueError` is raised).
 
-The codecs all use a similar interface.  Only deviation from the following
+The codecs all use a similar interface.  Only deviations from the following
 generic ones are documented for simplicity.
 
 
@@ -1171,7 +1171,7 @@ These are the UTF-16 codec APIs:
    ``1``, any byte order mark is copied to the output (where it will result in
    either a ``\ufeff`` or a ``\ufffe`` character).
 
-   After completion, *\*byteorder* is set to the current byte order at the end
+   After completion, ``*byteorder`` is set to the current byte order at the end
    of input data.
 
    If *byteorder* is ``NULL``, the codec starts in native order mode.
@@ -1302,7 +1302,7 @@ Character Map Codecs
 
 This codec is special in that it can be used to implement many different codecs
 (and this is in fact what was done to obtain most of the standard codecs
-included in the :mod:`encodings` package). The codec uses mapping to encode and
+included in the :mod:`encodings` package). The codec uses mappings to encode and
 decode characters.  The mapping objects provided must support the
 :meth:`__getitem__` mapping interface; dictionaries and sequences work well.
 
@@ -1426,7 +1426,7 @@ They all return ``NULL`` or ``-1`` if an exception occurs.
 .. c:function:: PyObject* PyUnicode_Splitlines(PyObject *s, int keepend)
 
    Split a Unicode string at line breaks, returning a list of Unicode strings.
-   CRLF is considered to be one line break.  If *keepend* is ``0``, the Line break
+   CRLF is considered to be one line break.  If *keepend* is ``0``, the line break
    characters are not included in the resulting strings.
 
 

Doc/c-api/veryhigh.rst

@@ -75,6 +75,8 @@ the same library that the Python runtime is using.
    :c:func:`PyRun_SimpleFile`.  *filename* is decoded from the filesystem
    encoding (:func:`sys.getfilesystemencoding`).  If *filename* is ``NULL``, this
    function uses ``"???"`` as the filename.
+   If *closeit* is true, the file is closed before
+   ``PyRun_SimpleFileExFlags()`` returns.
 
 
 .. c:function:: int PyRun_SimpleString(const char *command)

Doc/library/asyncio-task.rst

@@ -843,6 +843,9 @@ Task Object
       .. versionchanged:: 3.9
          Added the *msg* parameter.
 
+      .. versionchanged:: 3.11
+         The ``msg`` parameter is propagated from cancelled task to its awaiter.
+
       .. _asyncio_example_task_cancel:
 
       The following example illustrates how coroutines can intercept

Doc/library/configparser.rst

@@ -267,6 +267,9 @@ out.  Values can also span multiple lines, as long as they are indented deeper
 than the first line of the value.  Depending on the parser's mode, blank lines
 may be treated as parts of multiline values or ignored.
 
+By default,  a valid section name can be any string that does not contain '\\n' or ']'.
+To change this, see :attr:`ConfigParser.SECTCRE`.
+
 Configuration files may include comments, prefixed by specific
 characters (``#`` and ``;`` by default [1]_).  Comments may appear on
 their own on an otherwise empty line, possibly indented. [1]_

Doc/library/dis.rst

@@ -36,11 +36,12 @@ the following command can be used to display the disassembly of
    >>> dis.dis(myfunc)
      1           0 RESUME                   0
 
-     2           2 LOAD_GLOBAL              0 (len)
-                 4 LOAD_FAST                0 (alist)
-                 6 PRECALL_FUNCTION         1
-                 8 CALL                     0
-                10 RETURN_VALUE
+     2           2 PUSH_NULL
+                 4 LOAD_GLOBAL              0 (len)
+                 6 LOAD_FAST                0 (alist)
+                 8 PRECALL                  1
+                10 CALL                     1
+                12 RETURN_VALUE
 
 (The "2" is a line number).
 
@@ -106,9 +107,10 @@ Example::
     ...     print(instr.opname)
     ...
     RESUME
+    PUSH_NULL
     LOAD_GLOBAL
     LOAD_FAST
-    PRECALL_FUNCTION
+    PRECALL
     CALL
     RETURN_VALUE
 
@@ -1063,18 +1065,28 @@ iterations of the loop.
      with ``__cause__`` set to ``TOS``)
 
 
-.. opcode:: CALL (named)
+.. opcode:: CALL (argc)
 
-   Calls a callable object with the number of positional arguments specified by
-   the preceding :opcode:`PRECALL_FUNCTION` or :opcode:`PRECALL_METHOD` and
-   the named arguments specified by the preceding :opcode:`KW_NAMES`, if any.
-   *named* indicates the number of named arguments.
-   On the stack are (in ascending order):
+   Calls a callable object with the number of arguments specified by ``argc``,
+   including the named arguments specified by the preceding
+   :opcode:`KW_NAMES`, if any.
+   On the stack are (in ascending order), either:
 
+   * NULL
    * The callable
    * The positional arguments
    * The named arguments
 
+   or:
+
+   * The callable
+   * ``self``
+   * The remaining positional arguments
+   * The named arguments
+
+   ``argc`` is the total of the positional and named arguments, excluding
+   ``self`` when a ``NULL`` is not present.
+
    ``CALL`` pops all arguments and the callable object off the stack,
    calls the callable object with those arguments, and pushes the return value
    returned by the callable object.
@@ -1102,33 +1114,34 @@ iterations of the loop.
    Loads a method named ``co_names[namei]`` from the TOS object. TOS is popped.
    This bytecode distinguishes two cases: if TOS has a method with the correct
    name, the bytecode pushes the unbound method and TOS. TOS will be used as
-   the first argument (``self``) by :opcode:`PRECALL_METHOD` when calling the
+   the first argument (``self``) by :opcode:`CALL` when calling the
    unbound method. Otherwise, ``NULL`` and the object return by the attribute
    lookup are pushed.
 
    .. versionadded:: 3.7
 
 
-.. opcode:: PRECALL_METHOD (argc)
+.. opcode:: PRECALL (argc)
 
-   Prefixes :opcode:`CALL` (possibly with an intervening ``KW_NAMES``).
-   This opcode is designed to be used with :opcode:`LOAD_METHOD`.
-   Sets internal variables, so that :opcode:`CALL`
-   clean up after :opcode:`LOAD_METHOD` correctly.
+   Prefixes :opcode:`CALL`. Logically this is a no op.
+   It exists to enable effective specialization of calls.
+   ``argc`` is the number of arguments as described in :opcode:`CALL`.
 
    .. versionadded:: 3.11
 
 
-.. opcode:: PRECALL_FUNCTION (args)
+.. opcode:: PUSH_NULL
 
-   Prefixes :opcode:`CALL` (possibly with an intervening ``KW_NAMES``).
-   Sets internal variables, so that :opcode:`CALL` can execute correctly.
+    Pushes a ``NULL`` to the stack.
+    Used in the call sequence to match the ``NULL`` pushed by
+    :opcode:`LOAD_METHOD` for non-method calls.
 
    .. versionadded:: 3.11
 
 
 .. opcode:: KW_NAMES (i)
 
+   Prefixes :opcode:`PRECALL`.
    Stores a reference to ``co_consts[consti]`` into an internal variable
    for use by :opcode:`CALL`. ``co_consts[consti]`` must be a tuple of strings.
 

Doc/library/locale.rst

@@ -301,6 +301,8 @@ The :mod:`locale` module defines the following exception and functions:
    *language code* and *encoding* may be ``None`` if their values cannot be
    determined.
 
+   .. deprecated:: 3.11 3.13
+
 
 .. function:: getlocale(category=LC_CTYPE)
 

Doc/reference/compound_stmts.rst

@@ -154,17 +154,20 @@ The :keyword:`for` statement is used to iterate over the elements of a sequence
 (such as a string, tuple or list) or other iterable object:
 
 .. productionlist:: python-grammar
-   for_stmt: "for" `target_list` "in" `expression_list` ":" `suite`
+   for_stmt: "for" `target_list` "in" `starred_list` ":" `suite`
            : ["else" ":" `suite`]
 
 The expression list is evaluated once; it should yield an iterable object.  An
-iterator is created for the result of the ``expression_list``.  The suite is
-then executed once for each item provided by the iterator, in the order returned
-by the iterator.  Each item in turn is assigned to the target list using the
-standard rules for assignments (see :ref:`assignment`), and then the suite is
-executed.  When the items are exhausted (which is immediately when the sequence
-is empty or an iterator raises a :exc:`StopIteration` exception), the suite in
-the :keyword:`!else` clause, if present, is executed, and the loop terminates.
+iterator is created for the result of the ``starred_list``.  The expression
+list can contain starred elements (``*x, *y``) that will be unpacked in the
+final iterator (as when constructing a ``tuple`` or ``list`` literal). The
+suite is then executed once for each item provided by the iterator, in the
+order returned by the iterator.  Each item in turn is assigned to the target
+list using the standard rules for assignments (see :ref:`assignment`), and then
+the suite is executed.  When the items are exhausted (which is immediately when
+the sequence is empty or an iterator raises a :exc:`StopIteration` exception),
+the suite in the :keyword:`!else` clause, if present, is executed, and the loop
+terminates.
 
 .. index::
    statement: break
@@ -196,6 +199,8 @@ the built-in function :func:`range` returns an iterator of integers suitable to
 emulate the effect of Pascal's ``for i := a to b do``; e.g., ``list(range(3))``
 returns the list ``[0, 1, 2]``.
 
+.. versionchanged:: 3.11
+   Starred elements are now allowed in the expression list.
 
 .. _try:
 .. _except:

Doc/using/windows.rst

@@ -129,8 +129,8 @@ suppressing the UI in order to change some of the defaults.
 To completely hide the installer UI and install Python silently, pass the
 ``/quiet`` option. To skip past the user interaction but still display
 progress and errors, pass the ``/passive`` option. The ``/uninstall``
-option may be passed to immediately begin removing Python - no prompt will be
-displayed.
+option may be passed to immediately begin removing Python - no confirmation
+prompt will be displayed.
 
 All other options are passed as ``name=value``, where the value is usually
 ``0`` to disable a feature, ``1`` to enable a feature, or a path. The full list

Doc/whatsnew/3.11.rst

@@ -160,6 +160,9 @@ traceback. (Contributed by Irit Katriel in :issue:`45607`.)
 Other Language Changes
 ======================
 
+* Starred expressions can be used in :ref:`for statements<for>`. (See
+  :issue:`46725` for more details.)
+
 * Asynchronous comprehensions are now allowed inside comprehensions in
   asynchronous functions. Outer comprehensions implicitly become
   asynchronous. (Contributed by Serhiy Storchaka in :issue:`33346`.)
@@ -483,6 +486,12 @@ Deprecated
 
   (Contributed by Hugo van Kemenade in :issue:`45173`.)
 
+* The :func:`locale.getdefaultlocale` function is deprecated and will be
+  removed in Python 3.13. Use :func:`locale.setlocale`,
+  :func:`locale.getpreferredencoding(False) <locale.getpreferredencoding>` and
+  :func:`locale.getlocale` functions instead.
+  (Contributed by Victor Stinner in :issue:`46659`.)
+
 
 Removed
 =======
@@ -844,6 +853,8 @@ Porting to Python 3.11
     use ``PyObject_GetAttrString((PyObject*)frame, "f_locals")``.
   * ``f_lasti``: removed,
     use ``PyObject_GetAttrString((PyObject*)frame, "f_lasti")``.
+    Code using ``f_lasti`` with ``PyCode_Addr2Line()`` should use
+    :c:func:`PyFrame_GetLineNumber` instead.
 
   The following fields were removed entirely, as they were details
   of the old implementation:
@@ -1010,3 +1021,7 @@ Removed
   public C API by mistake, it must only be used by Python internally.
   Use the ``PyTypeObject.tp_members`` member instead.
   (Contributed by Victor Stinner in :issue:`40170`.)
+
+* Remove the ``HAVE_PY_SET_53BIT_PRECISION`` macro (moved to the internal C
+  API).
+  (Contributed by Victor Stinner in :issue:`45412`.)

Include/Python.h

@@ -39,6 +39,7 @@
 #include "pymacro.h"
 #include "pymath.h"
 #include "pymem.h"
+#include "pybuffer.h"
 #include "object.h"
 #include "objimpl.h"
 #include "typeslots.h"
@@ -50,7 +51,6 @@
 #include "longobject.h"
 #include "cpython/longintrepr.h"
 #include "boolobject.h"
-#include "buffer.h"
 #include "floatobject.h"
 #include "complexobject.h"
 #include "rangeobject.h"