From 81c870da693d85cde720b1cacbf36a000c6e90c5 Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Sun, 8 May 2022 15:33:06 -0700 Subject: [PATCH 1/6] gh-77024: test.support: Improve documentation This is a rework of #5774 on current main. I was a bit more conservative in making changes than the original PR. See @csabella's comments on issue #77024 and the discussion on #5774 for explanations of several of the changes. Co-authored-by: Cheryl Sabella --- Doc/library/test.rst | 103 +++++++++++++++++++++++++------------------ 1 file changed, 59 insertions(+), 44 deletions(-) diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 699db14596f250..af9fb0c17e5722 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -325,11 +325,6 @@ The :mod:`test.support` module defines the following constants: to make writes blocking. -.. data:: TEST_SUPPORT_DIR - - Set to the top level directory that contains :mod:`test.support`. - - .. data:: TEST_HOME_DIR Set to the top level directory for the test package. @@ -359,13 +354,13 @@ The :mod:`test.support` module defines the following constants: .. data:: MISSING_C_DOCSTRINGS - Return ``True`` if running on CPython, not on Windows, and configuration + Set to ``True`` if running on CPython, not on Windows, and configuration not set with ``WITH_DOC_STRINGS``. .. data:: HAVE_DOCSTRINGS - Check for presence of docstrings. + Set to ``True`` if docstrings are enabled. .. data:: TEST_HTTP_URL @@ -423,11 +418,6 @@ The :mod:`test.support` module defines the following functions: Used when tests are executed by :mod:`test.regrtest`. -.. function:: system_must_validate_cert(f) - - Raise :exc:`unittest.SkipTest` on TLS certification validation failures. - - .. function:: sortdict(dict) Return a repr of *dict* with keys sorted. @@ -445,12 +435,12 @@ The :mod:`test.support` module defines the following functions: .. function:: match_test(test) - Match *test* to patterns set in :func:`set_match_tests`. + Return whether *test* matches the patterns set in :func:`set_match_tests`. -.. function:: set_match_tests(patterns) +.. function:: set_match_tests(accept_patterns=None, ignore_patterns=None) - Define match test with regular expression *patterns*. + Define match patterns for filtering tests. .. function:: run_unittest(*classes) @@ -490,7 +480,9 @@ The :mod:`test.support` module defines the following functions: .. function:: check_impl_detail(**guards) Use this check to guard CPython's implementation-specific tests or to - run them only on the implementations guarded by the arguments:: + run them only on the implementations guarded by the arguments. This + function returns ``True`` or ``False`` depending on the host platform. + Example usage:: check_impl_detail() # Only on CPython (default). check_impl_detail(jython=True) # Only on Jython. @@ -509,7 +501,7 @@ The :mod:`test.support` module defines the following functions: time the regrtest began. -.. function:: get_original_stdout +.. function:: get_original_stdout() Return the original stdout set by :func:`record_original_stdout` or ``sys.stdout`` if it's not set. @@ -554,7 +546,7 @@ The :mod:`test.support` module defines the following functions: .. function:: disable_faulthandler() - A context manager that replaces ``sys.stderr`` with ``sys.__stderr__``. + A context manager that temporary disables :mod:`faulthandler`. .. function:: gc_collect() @@ -568,7 +560,7 @@ The :mod:`test.support` module defines the following functions: .. function:: disable_gc() A context manager that disables the garbage collector upon entry and - reenables it upon exit. + reenables it upon exit, if it was initially enabled. .. function:: swap_attr(obj, attr, new_val) @@ -642,14 +634,14 @@ The :mod:`test.support` module defines the following functions: .. function:: calcobjsize(fmt) - Return :func:`struct.calcsize` for ``nP{fmt}0n`` or, if ``gettotalrefcount`` - exists, ``2PnP{fmt}0P``. + Return the size of the :c:type:`PyObject` whose structure is + defined by *fmt* including the Python object header and alignment. .. function:: calcvobjsize(fmt) - Return :func:`struct.calcsize` for ``nPn{fmt}0n`` or, if ``gettotalrefcount`` - exists, ``2PnPn{fmt}0P``. + Return the size of the :c:type:`PyVarObject` whose structure is + defined by *fmt* including the Python object header and alignment. .. function:: checksizeof(test, o, size) @@ -665,6 +657,11 @@ The :mod:`test.support` module defines the following functions: have an associated comment identifying the relevant tracker issue. +.. function:: system_must_validate_cert(f) + + A decorator that skips the test on TLS certification validation failures. + + .. decorator:: run_with_locale(catstr, *locales) A decorator for running a function in a different locale, correctly @@ -682,19 +679,19 @@ The :mod:`test.support` module defines the following functions: .. decorator:: requires_freebsd_version(*min_version) Decorator for the minimum version when running test on FreeBSD. If the - FreeBSD version is less than the minimum, raise :exc:`unittest.SkipTest`. + FreeBSD version is less than the minimum, the test is skipped. .. decorator:: requires_linux_version(*min_version) Decorator for the minimum version when running test on Linux. If the - Linux version is less than the minimum, raise :exc:`unittest.SkipTest`. + Linux version is less than the minimum, the test is skipped. .. decorator:: requires_mac_version(*min_version) Decorator for the minimum version when running test on macOS. If the - macOS version is less than the minimum, raise :exc:`unittest.SkipTest`. + macOS version is less than the minimum, the test is skipped. .. decorator:: requires_IEEE_754 @@ -732,7 +729,7 @@ The :mod:`test.support` module defines the following functions: Decorator for only running the test if :data:`HAVE_DOCSTRINGS`. -.. decorator:: cpython_only(test) +.. decorator:: cpython_only Decorator for tests only applicable to CPython. @@ -743,12 +740,12 @@ The :mod:`test.support` module defines the following functions: returns ``False``, then uses *msg* as the reason for skipping the test. -.. decorator:: no_tracing(func) +.. decorator:: no_tracing Decorator to temporarily turn off tracing for the duration of the test. -.. decorator:: refcount_test(test) +.. decorator:: refcount_test Decorator for tests which involve reference counting. The decorator does not run the test if it is not run by CPython. Any trace function is unset @@ -771,10 +768,9 @@ The :mod:`test.support` module defines the following functions: means the test doesn't support dummy runs when ``-M`` is not specified. -.. decorator:: bigaddrspacetest(f) +.. decorator:: bigaddrspacetest - Decorator for tests that fill the address space. *f* is the function to - wrap. + Decorator for tests that fill the address space. .. function:: check_syntax_error(testcase, statement, errtext='', *, lineno=None, offset=None) @@ -876,7 +872,7 @@ The :mod:`test.support` module defines the following functions: .. function:: check_free_after_iterating(test, iter, cls, args=()) - Assert that *iter* is deallocated after iterating. + Assert instances of *cls* are deallocated after iterating. .. function:: missing_compiler_executable(cmd_names=[]) @@ -967,6 +963,16 @@ The :mod:`test.support` module defines the following classes: Class to save and restore signal handlers registered by the Python signal handler. + .. method:: save(self) + + Save the signal handlers to a dictionary mapping signal numbers to the + current signal handler. + + .. method:: restore(self) + + Set the signal numbers from the :meth:`save` dictionary to the saved + handler. + .. class:: Matcher() @@ -1110,11 +1116,11 @@ script execution tests. variables *env_vars* succeeds (``rc == 0``) and return a ``(return code, stdout, stderr)`` tuple. - If the ``__cleanenv`` keyword is set, *env_vars* is used as a fresh + If the *__cleanenv* keyword is set, *env_vars* is used as a fresh environment. Python is started in isolated mode (command line option ``-I``), - except if the ``__isolated`` keyword is set to ``False``. + except if the *__isolated* keyword is set to ``False``. .. versionchanged:: 3.9 The function no longer strips whitespaces from *stderr*. @@ -1225,15 +1231,17 @@ The :mod:`test.support.threading_helper` module provides support for threading t is still alive after *timeout* seconds. -.. decorator:: reap_threads(func) +.. decorator:: reap_threads Decorator to ensure the threads are cleaned up even if the test fails. .. function:: start_threads(threads, unlock=None) - Context manager to start *threads*. It attempts to join the threads upon - exit. + Context manager to start *threads*, which is a sequence of threads. + *unlock* is a function called after the threads are started, even if an + exception was raised; an example would be :meth:`threading.Event.set`. + This also attempts to join the started threads upon exit. .. function:: threading_cleanup(*original_values) @@ -1315,7 +1323,10 @@ The :mod:`test.support.os_helper` module provides support for os tests. .. data:: TESTFN_NONASCII - Set to a filename containing the :data:`FS_NONASCII` character. + Set to a filename containing the :data:`FS_NONASCII` character, if it exists. + This guarantees that if the filename exists, it can be encoded and decoded + with the default filesystem encoding. This allows tests that require a + non-ASCII filename to be easily skipped on platforms where they can't work. .. data:: TESTFN_UNENCODABLE @@ -1414,12 +1425,15 @@ The :mod:`test.support.os_helper` module provides support for os tests. Call :func:`os.rmdir` on *filename*. On Windows platforms, this is wrapped with a wait loop that checks for the existence of the file. + This is needed due to antivirus programs that can hold files open and prevent + deletion. .. function:: rmtree(path) Call :func:`shutil.rmtree` on *path* or call :func:`os.lstat` and - :func:`os.rmdir` to remove a path and its contents. On Windows platforms, + :func:`os.rmdir` to remove a path and its contents. As with :func:`rmdir`, + on Windows platforms this is wrapped with a wait loop that checks for the existence of the files. @@ -1466,7 +1480,8 @@ The :mod:`test.support.os_helper` module provides support for os tests. .. function:: unlink(filename) - Call :func:`os.unlink` on *filename*. On Windows platforms, this is + Call :func:`os.unlink` on *filename*. As with :func:`rmdir`, + on Windows platforms, this is wrapped with a wait loop that checks for the existence of the file. @@ -1523,7 +1538,7 @@ The :mod:`test.support.import_helper` module provides support for import tests. .. versionadded:: 3.1 -.. function:: import_module(name, deprecated=False, *, required_on()) +.. function:: import_module(name, deprecated=False, *, required_on=()) This function imports and returns the named module. Unlike a normal import, this function raises :exc:`unittest.SkipTest` if the module @@ -1565,7 +1580,7 @@ The :mod:`test.support.import_helper` module provides support for import tests. A context manager to force import to return a new module reference. This is useful for testing module-level behaviors, such as the emission of a - DeprecationWarning on import. Example usage:: + :exc:`DeprecationWarning` on import. Example usage:: with CleanImport('foo'): importlib.import_module('foo') # New reference. @@ -1573,7 +1588,7 @@ The :mod:`test.support.import_helper` module provides support for import tests. .. class:: DirsOnSysPath(*paths) - A context manager to temporarily add directories to sys.path. + A context manager to temporarily add directories to :data:`sys.path`. This makes a copy of :data:`sys.path`, appends any directories given as positional arguments, then reverts :data:`sys.path` to the copied From 41ed0639ddf0ccc5fee24a07db5f5fc93908bf0e Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Mon, 23 May 2022 06:21:20 -0700 Subject: [PATCH 2/6] Feedback from Victor --- Doc/library/test.rst | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/Doc/library/test.rst b/Doc/library/test.rst index af9fb0c17e5722..2231fc349164a5 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -325,6 +325,11 @@ The :mod:`test.support` module defines the following constants: to make writes blocking. +.. data:: TEST_SUPPORT_DIR + + Set to the top level directory that contains :mod:`test.support`. + + .. data:: TEST_HOME_DIR Set to the top level directory for the test package. @@ -354,13 +359,19 @@ The :mod:`test.support` module defines the following constants: .. data:: MISSING_C_DOCSTRINGS - Set to ``True`` if running on CPython, not on Windows, and configuration - not set with ``WITH_DOC_STRINGS``. + Set to ``True`` if Python is built without docstrings (the + :c:macro:`WITH_DOC_STRINGS` macro is not defined). + See the :option:`configure --without-doc-strings <--without-doc-strings>` option. + + See also the :data:`HAVE_DOCSTRINGS` variable. .. data:: HAVE_DOCSTRINGS - Set to ``True`` if docstrings are enabled. + Set to ``True`` if function docstrings are available. + See the :option:`python -OO <-O>` which strips docstrings of functions implemented in Python. + + See also the :data:`MISSING_C_DOCSTRINGS` variable. .. data:: TEST_HTTP_URL @@ -440,7 +451,7 @@ The :mod:`test.support` module defines the following functions: .. function:: set_match_tests(accept_patterns=None, ignore_patterns=None) - Define match patterns for filtering tests. + Define match patterns on test filenames and test method names for filtering tests. .. function:: run_unittest(*classes) @@ -634,13 +645,13 @@ The :mod:`test.support` module defines the following functions: .. function:: calcobjsize(fmt) - Return the size of the :c:type:`PyObject` whose structure is + Return the size of the :c:type:`PyObject` whose structure members are defined by *fmt* including the Python object header and alignment. .. function:: calcvobjsize(fmt) - Return the size of the :c:type:`PyVarObject` whose structure is + Return the size of the :c:type:`PyVarObject` whose structure members are defined by *fmt* including the Python object header and alignment. From ccadae677c74a7499bcf47db6d91e08fd81f32cc Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 25 May 2022 19:54:05 -0700 Subject: [PATCH 3/6] Apply suggestions from code review Co-authored-by: Alex Waygood --- Doc/library/test.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 2231fc349164a5..b54989fad1ecad 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -369,7 +369,7 @@ The :mod:`test.support` module defines the following constants: .. data:: HAVE_DOCSTRINGS Set to ``True`` if function docstrings are available. - See the :option:`python -OO <-O>` which strips docstrings of functions implemented in Python. + See the :option:`python -OO <-O>` option, which strips docstrings of functions implemented in Python. See also the :data:`MISSING_C_DOCSTRINGS` variable. @@ -446,7 +446,7 @@ The :mod:`test.support` module defines the following functions: .. function:: match_test(test) - Return whether *test* matches the patterns set in :func:`set_match_tests`. + Determine whether *test* matches the patterns set in :func:`set_match_tests`. .. function:: set_match_tests(accept_patterns=None, ignore_patterns=None) @@ -646,7 +646,7 @@ The :mod:`test.support` module defines the following functions: .. function:: calcobjsize(fmt) Return the size of the :c:type:`PyObject` whose structure members are - defined by *fmt* including the Python object header and alignment. + defined by *fmt*. The returned value includes the size of the Python object header and alignment. .. function:: calcvobjsize(fmt) @@ -670,7 +670,7 @@ The :mod:`test.support` module defines the following functions: .. function:: system_must_validate_cert(f) - A decorator that skips the test on TLS certification validation failures. + A decorator that skips the decorated test on TLS certification validation failures. .. decorator:: run_with_locale(catstr, *locales) @@ -1252,7 +1252,7 @@ The :mod:`test.support.threading_helper` module provides support for threading t Context manager to start *threads*, which is a sequence of threads. *unlock* is a function called after the threads are started, even if an exception was raised; an example would be :meth:`threading.Event.set`. - This also attempts to join the started threads upon exit. + ``start_threads`` will attempt to join the started threads upon exit. .. function:: threading_cleanup(*original_values) @@ -1435,8 +1435,8 @@ The :mod:`test.support.os_helper` module provides support for os tests. .. function:: rmdir(filename) Call :func:`os.rmdir` on *filename*. On Windows platforms, this is - wrapped with a wait loop that checks for the existence of the file. - This is needed due to antivirus programs that can hold files open and prevent + wrapped with a wait loop that checks for the existence of the file, + which is needed due to antivirus programs that can hold files open and prevent deletion. From 212e267c8ca4e0c331346e7dd73a4c7f8662b70d Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 25 May 2022 19:54:38 -0700 Subject: [PATCH 4/6] Update Doc/library/test.rst --- Doc/library/test.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/test.rst b/Doc/library/test.rst index b54989fad1ecad..d520f5f8951b70 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -652,7 +652,7 @@ The :mod:`test.support` module defines the following functions: .. function:: calcvobjsize(fmt) Return the size of the :c:type:`PyVarObject` whose structure members are - defined by *fmt* including the Python object header and alignment. + defined by *fmt*. The returned value includes the size of the Python object header and alignment. .. function:: checksizeof(test, o, size) From 66199b4a396e200e42c86ab0529e67a9090c0bce Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Wed, 25 May 2022 19:58:01 -0700 Subject: [PATCH 5/6] one more --- Doc/library/test.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/test.rst b/Doc/library/test.rst index d520f5f8951b70..325adc7cff8885 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -570,8 +570,8 @@ The :mod:`test.support` module defines the following functions: .. function:: disable_gc() - A context manager that disables the garbage collector upon entry and - reenables it upon exit, if it was initially enabled. + A context manager that disables the garbage collector on entry. On + exit, the garbage collector is restored to its prior state. .. function:: swap_attr(obj, attr, new_val) From 36981dd02890905102b3b98c7b24e43785aecd1c Mon Sep 17 00:00:00 2001 From: Jelle Zijlstra Date: Thu, 26 May 2022 21:46:17 -0700 Subject: [PATCH 6/6] Apply suggestions from code review Co-authored-by: Alex Waygood --- Doc/library/test.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 325adc7cff8885..72411007e24528 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -1127,11 +1127,11 @@ script execution tests. variables *env_vars* succeeds (``rc == 0``) and return a ``(return code, stdout, stderr)`` tuple. - If the *__cleanenv* keyword is set, *env_vars* is used as a fresh + If the *__cleanenv* keyword-only parameter is set, *env_vars* is used as a fresh environment. Python is started in isolated mode (command line option ``-I``), - except if the *__isolated* keyword is set to ``False``. + except if the *__isolated* keyword-only parameter is set to ``False``. .. versionchanged:: 3.9 The function no longer strips whitespaces from *stderr*.