Merge branch 'main' into typewatch

* main: (38 commits)
  pythongh-92886: make test_ast pass with -O (assertions off) (pythonGH-98058)
  pythongh-92886: make test_coroutines pass with -O (assertions off) (pythonGH-98060)
  pythongh-57179: Add note on symlinks for os.walk (python#94799)
  pythongh-94808: Fix regex on exotic platforms (python#98036)
  pythongh-90085: Remove vestigial -t and -c timeit options (python#94941)
  pythonGH-83901: Improve Signature.bind error message for missing keyword-only params (python#95347)
  pythongh-61105: Add default param, note on using cookiejar subclass (python#95427)
  pythongh-96288: Add a sentence to `os.mkdir`'s docstring. (python#96271)
  pythongh-96073: fix backticks in NEWS entry (pythonGH-98056)
  pythongh-92886: [clinic.py] raise exception on invalid input instead of assertion (pythonGH-98051)
  pythongh-97997: Add col_offset field to tokenizer and use that for AST nodes (python#98000)
  pythonGH-88968: Reject socket that is already used as a transport (python#98010)
  pythongh-96346: Use double caching for re._compile() (python#96347)
  pythongh-91708: Revert params note in urllib.parse.urlparse table (python#96699)
  pythongh-96265: Fix some formatting in faq/design.rst (python#96924)
  pythongh-73196: Add namespace/scope clarification for inheritance section (python#92840)
  pythongh-97646: Change `.js` and `.mjs` files mimetype to conform to RFC 9239 (python#97934)
  pythongh-97923: Always run Ubuntu SSL tests with others in CI (python#97940)
  pythongh-97956: Mention `generate_global_objects.py` in `AC How-To` (python#97957)
  pythongh-96959: Update HTTP links which are redirected to HTTPS (python#98039)
  ...

Author: carljm

.github/workflows/build.yml

@@ -31,15 +31,13 @@ jobs:
     runs-on: ubuntu-latest
     outputs:
       run_tests: ${{ steps.check.outputs.run_tests }}
-      run_ssl_tests: ${{ steps.check.outputs.run_ssl_tests }}
     steps:
       - uses: actions/checkout@v3
       - name: Check for source changes
         id: check
         run: |
           if [ -z "$GITHUB_BASE_REF" ]; then
             echo '::set-output name=run_tests::true'
-            echo '::set-output name=run_ssl_tests::true'
           else
             git fetch origin $GITHUB_BASE_REF --depth=1
             # git diff "origin/$GITHUB_BASE_REF..." (3 dots) may be more
@@ -56,7 +54,6 @@ jobs:
             #
             # https://github.com/python/core-workflow/issues/373
             git diff --name-only origin/$GITHUB_BASE_REF.. | grep -qvE '(\.rst$|^Doc|^Misc)' && echo '::set-output name=run_tests::true' || true
-            git diff --name-only origin/$GITHUB_BASE_REF.. | grep -qE '(ssl|hashlib|hmac|^.github)' && echo '::set-output name=run_ssl_tests::true' || true
           fi
 
   check_generated_files:
@@ -230,7 +227,7 @@ jobs:
     name: 'Ubuntu SSL tests with OpenSSL'
     runs-on: ubuntu-20.04
     needs: check_source
-    if: needs.check_source.outputs.run_tests == 'true' && needs.check_source.outputs.run_ssl_tests == 'true'
+    if: needs.check_source.outputs.run_tests == 'true'
     strategy:
       fail-fast: false
       matrix:

Doc/c-api/conversion.rst

@@ -28,7 +28,8 @@ not.
 The wrappers ensure that ``str[size-1]`` is always ``'\0'`` upon return. They
 never write more than *size* bytes (including the trailing ``'\0'``) into str.
 Both functions require that ``str != NULL``, ``size > 0``, ``format != NULL``
-and ``size < INT_MAX``.
+and ``size < INT_MAX``. Note that this means there is no equivalent to the C99
+``n = snprintf(NULL, 0, ...)`` which would determine the necessary buffer size.
 
 The return value (*rv*) for these functions should be interpreted as follows:
 

Doc/faq/design.rst

@@ -129,7 +129,7 @@ reference or call the method from a particular class.  In C++, if you want to
 use a method from a base class which is overridden in a derived class, you have
 to use the ``::`` operator -- in Python you can write
 ``baseclass.methodname(self, <argument list>)``.  This is particularly useful
-for :meth:`__init__` methods, and in general in cases where a derived class
+for :meth:`~object.__init__` methods, and in general in cases where a derived class
 method wants to extend the base class method of the same name and thus has to
 call the base class method somehow.
 
@@ -232,7 +232,8 @@ Similar methods exist for bytes and bytearray objects.
 How fast are exceptions?
 ------------------------
 
-A try/except block is extremely efficient if no exceptions are raised.  Actually
+A :keyword:`try`/:keyword:`except` block is extremely efficient if no exceptions
+are raised.  Actually
 catching an exception is expensive.  In versions of Python prior to 2.0 it was
 common to use this idiom::
 
@@ -352,7 +353,7 @@ will probably run out of file descriptors::
        c = f.read(1)
 
 Indeed, using CPython's reference counting and destructor scheme, each new
-assignment to *f* closes the previous file.  With a traditional GC, however,
+assignment to ``f`` closes the previous file.  With a traditional GC, however,
 those file objects will only get collected (and closed) at varying and possibly
 long intervals.
 
@@ -376,10 +377,10 @@ Python to work with it.)
 
 Traditional GC also becomes a problem when Python is embedded into other
 applications.  While in a standalone Python it's fine to replace the standard
-malloc() and free() with versions provided by the GC library, an application
-embedding Python may want to have its *own* substitute for malloc() and free(),
+``malloc()`` and ``free()`` with versions provided by the GC library, an application
+embedding Python may want to have its *own* substitute for ``malloc()`` and ``free()``,
 and may not want Python's.  Right now, CPython works with anything that
-implements malloc() and free() properly.
+implements ``malloc()`` and ``free()`` properly.
 
 
 Why isn't all memory freed when CPython exits?
@@ -401,14 +402,15 @@ Why are there separate tuple and list data types?
 
 Lists and tuples, while similar in many respects, are generally used in
 fundamentally different ways.  Tuples can be thought of as being similar to
-Pascal records or C structs; they're small collections of related data which may
+Pascal ``records`` or C ``structs``; they're small collections of related data which may
 be of different types which are operated on as a group.  For example, a
 Cartesian coordinate is appropriately represented as a tuple of two or three
 numbers.
 
 Lists, on the other hand, are more like arrays in other languages.  They tend to
 hold a varying number of objects all of which have the same type and which are
-operated on one-by-one.  For example, ``os.listdir('.')`` returns a list of
+operated on one-by-one.  For example, :func:`os.listdir('.') <os.listdir>`
+returns a list of
 strings representing the files in the current directory.  Functions which
 operate on this output would generally not break if you added another file or
 two to the directory.
@@ -444,9 +446,9 @@ far) under most circumstances, and the implementation is simpler.
 
 Dictionaries work by computing a hash code for each key stored in the dictionary
 using the :func:`hash` built-in function.  The hash code varies widely depending
-on the key and a per-process seed; for example, "Python" could hash to
--539294296 while "python", a string that differs by a single bit, could hash
-to 1142331976.  The hash code is then used to calculate a location in an
+on the key and a per-process seed; for example, ``'Python'`` could hash to
+``-539294296`` while ``'python'``, a string that differs by a single bit, could hash
+to ``1142331976``.  The hash code is then used to calculate a location in an
 internal array where the value will be stored.  Assuming that you're storing
 keys that all have different hash values, this means that dictionaries take
 constant time -- O(1), in Big-O notation -- to retrieve a key.
@@ -497,7 +499,8 @@ Some unacceptable solutions that have been proposed:
 
 There is a trick to get around this if you need to, but use it at your own risk:
 You can wrap a mutable structure inside a class instance which has both a
-:meth:`__eq__` and a :meth:`__hash__` method.  You must then make sure that the
+:meth:`~object.__eq__` and a :meth:`~object.__hash__` method.
+You must then make sure that the
 hash value for all such wrapper objects that reside in a dictionary (or other
 hash based structure), remain fixed while the object is in the dictionary (or
 other structure). ::
@@ -528,7 +531,7 @@ is True``) then ``hash(o1) == hash(o2)`` (ie, ``o1.__hash__() == o2.__hash__()``
 regardless of whether the object is in a dictionary or not.  If you fail to meet
 these restrictions dictionaries and other hash based structures will misbehave.
 
-In the case of ListWrapper, whenever the wrapper object is in a dictionary the
+In the case of :class:`!ListWrapper`, whenever the wrapper object is in a dictionary the
 wrapped list must not change to avoid anomalies.  Don't do this unless you are
 prepared to think hard about the requirements and the consequences of not
 meeting them correctly.  Consider yourself warned.
@@ -581,9 +584,9 @@ exhaustive test suites that exercise every line of code in a module.
 An appropriate testing discipline can help build large complex applications in
 Python as well as having interface specifications would.  In fact, it can be
 better because an interface specification cannot test certain properties of a
-program.  For example, the :meth:`append` method is expected to add new elements
+program.  For example, the :meth:`list.append` method is expected to add new elements
 to the end of some internal list; an interface specification cannot test that
-your :meth:`append` implementation will actually do this correctly, but it's
+your :meth:`list.append` implementation will actually do this correctly, but it's
 trivial to check this property in a test suite.
 
 Writing test suites is very helpful, and you might want to design your code to
@@ -599,14 +602,14 @@ Why is there no goto?
 In the 1970s people realized that unrestricted goto could lead
 to messy "spaghetti" code that was hard to understand and revise.
 In a high-level language, it is also unneeded as long as there
-are ways to branch (in Python, with ``if`` statements and ``or``,
-``and``, and ``if-else`` expressions) and loop (with ``while``
-and ``for`` statements, possibly containing ``continue`` and ``break``).
+are ways to branch (in Python, with :keyword:`if` statements and :keyword:`or`,
+:keyword:`and`, and :keyword:`if`/:keyword:`else` expressions) and loop (with :keyword:`while`
+and :keyword:`for` statements, possibly containing :keyword:`continue` and :keyword:`break`).
 
 One can also use exceptions to provide a "structured goto"
 that works even across
 function calls.  Many feel that exceptions can conveniently emulate all
-reasonable uses of the "go" or "goto" constructs of C, Fortran, and other
+reasonable uses of the ``go`` or ``goto`` constructs of C, Fortran, and other
 languages.  For example::
 
    class label(Exception): pass  # declare a label
@@ -620,7 +623,7 @@ languages.  For example::
    ...
 
 This doesn't allow you to jump into the middle of a loop, but that's usually
-considered an abuse of goto anyway.  Use sparingly.
+considered an abuse of ``goto`` anyway.  Use sparingly.
 
 
 Why can't raw strings (r-strings) end with a backslash?
@@ -652,7 +655,7 @@ If you're trying to build a pathname for a DOS command, try e.g. one of ::
 Why doesn't Python have a "with" statement for attribute assignments?
 ---------------------------------------------------------------------
 
-Python has a 'with' statement that wraps the execution of a block, calling code
+Python has a :keyword:`with` statement that wraps the execution of a block, calling code
 on the entrance and exit from the block.  Some languages have a construct that
 looks like this::
 
@@ -679,13 +682,13 @@ For instance, take the following incomplete snippet::
        with a:
            print(x)
 
-The snippet assumes that "a" must have a member attribute called "x".  However,
+The snippet assumes that ``a`` must have a member attribute called ``x``. However,
 there is nothing in Python that tells the interpreter this. What should happen
-if "a" is, let us say, an integer?  If there is a global variable named "x",
-will it be used inside the with block?  As you see, the dynamic nature of Python
+if ``a`` is, let us say, an integer?  If there is a global variable named ``x``,
+will it be used inside the :keyword:`with` block?  As you see, the dynamic nature of Python
 makes such choices much harder.
 
-The primary benefit of "with" and similar language features (reduction of code
+The primary benefit of :keyword:`with` and similar language features (reduction of code
 volume) can, however, easily be achieved in Python by assignment.  Instead of::
 
    function(args).mydict[index][index].a = 21
@@ -714,7 +717,8 @@ Why don't generators support the with statement?
 For technical reasons, a generator used directly as a context manager
 would not work correctly.  When, as is most common, a generator is used as
 an iterator run to completion, no closing is needed.  When it is, wrap
-it as "contextlib.closing(generator)" in the 'with' statement.
+it as :func:`contextlib.closing(generator) <contextlib.closing>`
+in the :keyword:`with` statement.
 
 
 Why are colons required for the if/while/def/class statements?

Doc/faq/library.rst

@@ -609,7 +609,7 @@ use ``p.read(n)``.
    substituted for standard input and output.  You will have to use pseudo ttys
    ("ptys") instead of pipes. Or you can use a Python interface to Don Libes'
    "expect" library.  A Python extension that interfaces to expect is called
-   "expy" and available from http://expectpy.sourceforge.net.  A pure Python
+   "expy" and available from https://expectpy.sourceforge.net.  A pure Python
    solution that works like expect is `pexpect
    <https://pypi.org/project/pexpect/>`_.
 

Doc/howto/clinic.rst

@@ -539,7 +539,15 @@ Let's dive in!
         };
 
 
-16. Compile, then run the relevant portions of the regression-test suite.
+16. Argument Clinic may generate new instances of ``_Py_ID``. For example::
+
+        &_Py_ID(new_unique_py_id)
+
+    If it does, you'll have to run ``Tools/scripts/generate_global_objects.py``
+    to regenerate the list of precompiled identifiers at this point.
+
+
+17. Compile, then run the relevant portions of the regression-test suite.
     This change should not introduce any new compile-time warnings or errors,
     and there should be no externally visible change to Python's behavior.
 

Tools/scripts/diff.py renamed to Doc/includes/diff.py

@@ -1,4 +1,3 @@
-#!/usr/bin/env python3
 """ Command line interface to difflib.py providing diffs in four formats:
 
 * ndiff:    lists every line and highlights interline changes.

Tools/scripts/ndiff.py renamed to Doc/includes/ndiff.py

@@ -1,16 +1,3 @@
-#! /usr/bin/env python3
-
-# Module ndiff version 1.7.0
-# Released to the public domain 08-Dec-2000,
-# by Tim Peters ([email protected]).
-
-# Provided as-is; use at your own risk; no warranty; no promises; enjoy!
-
-# ndiff.py is now simply a front-end to the difflib.ndiff() function.
-# Originally, it contained the difflib.SequenceMatcher class as well.
-# This completes the raiding of reusable code from this formerly
-# self-contained script.
-
 """ndiff [-q] file1 file2
     or
 ndiff (-r1 | -r2) < ndiff_output > file1_or_file2
@@ -121,13 +108,4 @@ def restore(which):
     sys.stdout.writelines(restored)
 
 if __name__ == '__main__':
-    args = sys.argv[1:]
-    if "-profile" in args:
-        import profile, pstats
-        args.remove("-profile")
-        statf = "ndiff.pro"
-        profile.run("main(args)", statf)
-        stats = pstats.Stats(statf)
-        stats.strip_dirs().sort_stats('time').print_stats()
-    else:
-        main(args)
+    main(sys.argv[1:])

Doc/includes/tzinfo_examples.py

@@ -71,7 +71,7 @@ def first_sunday_on_or_after(dt):
 # DST start and end times. For a complete and up-to-date set of DST rules
 # and timezone definitions, visit the Olson Database (or try pytz):
 # http://www.twinsun.com/tz/tz-link.htm
-# http://sourceforge.net/projects/pytz/ (might not be up-to-date)
+# https://sourceforge.net/projects/pytz/ (might not be up-to-date)
 #
 # In the US, since 2007, DST starts at 2am (standard time) on the second
 # Sunday in March, which is the first Sunday on or after Mar 8.

Doc/library/csv.rst

@@ -420,7 +420,7 @@ Dialects support the following attributes:
 
 .. attribute:: Dialect.skipinitialspace
 
-   When :const:`True`, whitespace immediately following the *delimiter* is ignored.
+   When :const:`True`, spaces immediately following the *delimiter* are ignored.
    The default is :const:`False`.
 
 

Doc/library/difflib.rst

@@ -145,8 +145,6 @@ diffs. For comparing directories and files, see also, the :mod:`filecmp` module.
       The arguments for this method are the same as those for the :meth:`make_file`
       method.
 
-   :file:`Tools/scripts/diff.py` is a command-line front-end to this class and
-   contains a good example of its use.
 
 
 .. function:: context_diff(a, b, fromfile='', tofile='', fromfiledate='', tofiledate='', n=3, lineterm='\n')
@@ -240,8 +238,6 @@ diffs. For comparing directories and files, see also, the :mod:`filecmp` module.
    function :func:`IS_CHARACTER_JUNK`, which filters out whitespace characters (a
    blank or tab; it's a bad idea to include newline in this!).
 
-   :file:`Tools/scripts/ndiff.py` is a command-line front-end to this function.
-
       >>> diff = ndiff('one\ntwo\nthree\n'.splitlines(keepends=True),
       ...              'ore\ntree\nemu\n'.splitlines(keepends=True))
       >>> print(''.join(diff), end="")
@@ -759,7 +755,12 @@ A command-line interface to difflib
 -----------------------------------
 
 This example shows how to use difflib to create a ``diff``-like utility.
-It is also contained in the Python source distribution, as
-:file:`Tools/scripts/diff.py`.
 
-.. literalinclude:: ../../Tools/scripts/diff.py
+.. literalinclude:: ../includes/diff.py
+
+ndiff example
+-------------
+
+This example shows how to use :func:`difflib.ndiff`.
+
+.. literalinclude:: ../includes/ndiff.py