From 41bab32c21b259285db5901b5ceead8ede50f040 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Mon, 29 Aug 2022 23:34:50 +0200 Subject: [PATCH 1/8] gh-96143: Improve perf profiler docs --- Doc/howto/perf_profiling.rst | 45 ++++++++++--------- Doc/library/sys.rst | 29 ++++++++++++ Doc/using/cmdline.rst | 16 ++++--- Doc/whatsnew/3.12.rst | 21 +++++++++ ...2-08-20-18-36-40.gh-issue-96143.nh3GFM.rst | 11 +++-- Python/clinic/sysmodule.c.h | 8 ++-- Python/sysmodule.c | 12 ++--- 7 files changed, 100 insertions(+), 42 deletions(-) diff --git a/Doc/howto/perf_profiling.rst b/Doc/howto/perf_profiling.rst index ed8de888b3bc21..d820d1701deb01 100644 --- a/Doc/howto/perf_profiling.rst +++ b/Doc/howto/perf_profiling.rst @@ -25,7 +25,7 @@ fly before the execution of every Python function and it will teach ``perf`` the relationship between this piece of code and the associated Python function using `perf map files`_. -.. warning:: +.. note:: Support for the ``perf`` profiler is only currently available for Linux on selected architectures. Check the output of the configure build step or @@ -51,11 +51,11 @@ For example, consider the following script: if __name__ == "__main__": baz(1000000) -We can run perf to sample CPU stack traces at 9999 Hertz: +We can run ``perf`` to sample CPU stack traces at 9999 Hertz:: $ perf record -F 9999 -g -o perf.data python my_script.py -Then we can use perf report to analyze the data: +Then we can use ``perf`` report to analyze the data: .. code-block:: shell-session @@ -101,7 +101,7 @@ As you can see here, the Python functions are not shown in the output, only ``_P functions use the same C function to evaluate bytecode so we cannot know which Python function corresponds to which bytecode-evaluating function. -Instead, if we run the same experiment with perf support activated we get: +Instead, if we run the same experiment with ``perf`` support activated we get: .. code-block:: shell-session @@ -147,19 +147,27 @@ Instead, if we run the same experiment with perf support activated we get: -Enabling perf profiling mode ----------------------------- +How to enable the ``perf`` profiling mode +----------------------------------------- -There are two main ways to activate the perf profiling mode. If you want it to be -active since the start of the Python interpreter, you can use the `-Xperf` option: +There are three ways to activate the ``perf`` profiling mode: +using the :option:`-Xperf <-X>` command-line option, +using the :envvar:`PYTHONPERFSUPPORT` environment variable, +and using the :func:`sys.activate_stack_trampoline` and +:func:`sys.deactivate_stack_trampoline` APIs. +If you want profiling to be active when you start the Python interpreter, +use the :option:`-Xperf <-X>` option:: $ python -Xperf my_script.py -You can also set the :envvar:`PYTHONPERFSUPPORT` to a nonzero value to actiavate perf -profiling mode globally. +If you need to activate the ``perf`` profiling mode globally, +set the environment variable :envvar:`PYTHONPERFSUPPORT` +to a nonzero value. -There is also support for dynamically activating and deactivating the perf -profiling mode by using the APIs in the :mod:`sys` module: +If you need to dynamically activate and deactivate the ``perf`` profiling mode +in response to a signal or other communication mechanisms with your process, +use the :func:`sys.activate_stack_trampoline` and +:func:`sys.deactivate_stack_trampoline` APIs: .. code-block:: python @@ -172,27 +180,22 @@ profiling mode by using the APIs in the :mod:`sys` module: # Perf profiling is not active anymore -These APIs can be handy if you want to activate/deactivate profiling mode in -response to a signal or other communication mechanism with your process. - - - -Now we can analyze the data with ``perf report``: +Now we can analyze the data with ``perf report``:: $ perf report -g -i perf.data How to obtain the best results -------------------------------- +------------------------------ For the best results, Python should be compiled with ``CFLAGS="-fno-omit-frame-pointer -mno-omit-leaf-frame-pointer"`` as this allows profilers to unwind using only the frame pointer and not on DWARF debug -information. This is because as the code that is interposed to allow perf +information. This is because as the code that is interposed to allow ``perf`` support is dynamically generated it doesn't have any DWARF debugging information available. -You can check if you system has been compiled with this flag by running: +You can check if you system has been compiled with this flag by running:: $ python -m sysconfig | grep 'no-omit-frame-pointer' diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 43db4baf62dfb3..ba982409a82dae 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1524,6 +1524,35 @@ always available. This function has been added on a provisional basis (see :pep:`411` for details.) Use it only for debugging purposes. +.. function:: activate_stack_trampoline(backend, /) + + Activate the stack profiler trampoline *backend*. + The only supported backend is `"perf"`. + + .. availability:: Linux. + + .. versionadded:: 3.12 + + .. seealso:: + + :ref:`perf_profiling` + +.. function:: deactivate_stack_trampoline() + + Deactivate the stack profiler trampoline. + + .. availability:: Linux. + + .. versionadded:: 3.12 + +.. function:: is_stack_trampoline_active() + + Return ``True`` if the stack profiler trampoline is active. + + .. availability:: Linux. + + .. versionadded:: 3.12 + .. function:: _enablelegacywindowsfsencoding() Changes the :term:`filesystem encoding and error handler` to 'mbcs' and diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index fa2b07e468b3b5..e6c0e5cfa63cfa 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -535,12 +535,13 @@ Miscellaneous options development (running from the source tree) then the default is "off". Note that the "importlib_bootstrap" and "importlib_bootstrap_external" frozen modules are always used, even if this flag is set to "off". - * ``-X perf`` to activate compatibility mode with the ``perf`` profiler. + * ``-X perf`` activates compatibility mode with the ``perf`` profiler. When this option is activated, the Linux ``perf`` profiler will be able to report Python calls. This option is only available on some platforms and will do nothing if is not supported on the current system. The default value - is "off". See also :envvar:`PYTHONPERFSUPPORT` and :ref:`perf_profiling` - for more information. + is "off". See :ref:`perf_profiling` for more details. + This option is equivalent to setting the environment variable + :envvar:`PYTHONPERFSUPPORT` to ``1``. It also allows passing arbitrary values and retrieving them through the :data:`sys._xoptions` dictionary. @@ -1036,8 +1037,13 @@ conflict. .. envvar:: PYTHONPERFSUPPORT If this variable is set to a nonzero value, it activates compatibility mode - with the ``perf`` profiler so Python calls can be detected by it. See the - :ref:`perf_profiling` section for more information. + with the ``perf`` profiler so Python calls can be detected by it. + This is exactly equivalent to setting :option:`-X perf <-X>` + on the command line. + + If set to ``0``, disable ``perf`` profiler support. + + See :ref:`perf_profiling` for more details. .. versionadded:: 3.12 diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index f9fa8ac3123198..f8627164714b78 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -74,6 +74,14 @@ Important deprecations, removals or restrictions: New Features ============ +* Add :ref:`perf_profiling` through the new command-line option + :option:`-X perf <-X>`, as well as the new + :func:`sys.activate_stack_trampoline`, + :func:`sys.deactivate_stack_trampoline`, + and :func:`sys.is_stack_trampoline_active` APIs. + (Contributed by Pablo Galindo and Christian Heimes + with contributions from Gregory P. Smith [Google] and Mark Shannon + in :gh:`96123`.) Other Language Changes @@ -126,6 +134,19 @@ sqlite3 (Contributed by Erlend E. Aasland in :gh:`77617`.) +sys +--- + +* Add :func:`sys.activate_stack_trampoline` and + :func:`sys.deactivate_stack_trampoline` for activating and deactivating + stack profiler trampolines, + and :func:`sys.is_stack_trampoline_active` for querying if stack profiler + trampolines are active. + (Contributed by Pablo Galindo and Christian Heimes + with contributions from Gregory P. Smith [Google] and Mark Shannon + in :gh:`96123`.) + + Optimizations ============= diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-08-20-18-36-40.gh-issue-96143.nh3GFM.rst b/Misc/NEWS.d/next/Core and Builtins/2022-08-20-18-36-40.gh-issue-96143.nh3GFM.rst index 30f44fd453a547..377816bc31eab9 100644 --- a/Misc/NEWS.d/next/Core and Builtins/2022-08-20-18-36-40.gh-issue-96143.nh3GFM.rst +++ b/Misc/NEWS.d/next/Core and Builtins/2022-08-20-18-36-40.gh-issue-96143.nh3GFM.rst @@ -1,7 +1,6 @@ -Add a new ``-X perf`` Python command line option as well as -:func:`sys.activate_stack_trampoline` and :func:`sys.deactivate_stack_trampoline` -function in the :mod:`sys` module that allows to set/unset the interpreter in a -way that the Linux ``perf`` profiler can detect Python calls. The new -:func:`sys.is_stack_trampoline_active` function allows to query the state of the -perf trampoline. Design by Pablo Galindo. Patch by Pablo Galindo and Christian Heimes +Add :ref:`perf_profiling` through the new command-line option +:option:`-X perf <-X>`, as well as the new +:func:`sys.activate_stack_trampoline`, :func:`sys.deactivate_stack_trampoline`, +and :func:`sys.is_stack_trampoline_active` APIs. +Design by Pablo Galindo. Patch by Pablo Galindo and Christian Heimes with contributions from Gregory P. Smith [Google] and Mark Shannon. diff --git a/Python/clinic/sysmodule.c.h b/Python/clinic/sysmodule.c.h index ddf01a7ccdda08..b455c83e43f728 100644 --- a/Python/clinic/sysmodule.c.h +++ b/Python/clinic/sysmodule.c.h @@ -1155,7 +1155,7 @@ PyDoc_STRVAR(sys_activate_stack_trampoline__doc__, "activate_stack_trampoline($module, backend, /)\n" "--\n" "\n" -"Activate the perf profiler trampoline."); +"Activate stack profiler trampoline *backend*."); #define SYS_ACTIVATE_STACK_TRAMPOLINE_METHODDEF \ {"activate_stack_trampoline", (PyCFunction)sys_activate_stack_trampoline, METH_O, sys_activate_stack_trampoline__doc__}, @@ -1192,7 +1192,7 @@ PyDoc_STRVAR(sys_deactivate_stack_trampoline__doc__, "deactivate_stack_trampoline($module, /)\n" "--\n" "\n" -"Dectivate the perf profiler trampoline."); +"Dectivate stack profiler trampoline backend."); #define SYS_DEACTIVATE_STACK_TRAMPOLINE_METHODDEF \ {"deactivate_stack_trampoline", (PyCFunction)sys_deactivate_stack_trampoline, METH_NOARGS, sys_deactivate_stack_trampoline__doc__}, @@ -1210,7 +1210,7 @@ PyDoc_STRVAR(sys_is_stack_trampoline_active__doc__, "is_stack_trampoline_active($module, /)\n" "--\n" "\n" -"Returns *True* if the perf profiler trampoline is active."); +"Return *True* if the stack profiler trampoline is active."); #define SYS_IS_STACK_TRAMPOLINE_ACTIVE_METHODDEF \ {"is_stack_trampoline_active", (PyCFunction)sys_is_stack_trampoline_active, METH_NOARGS, sys_is_stack_trampoline_active__doc__}, @@ -1267,4 +1267,4 @@ sys_is_stack_trampoline_active(PyObject *module, PyObject *Py_UNUSED(ignored)) #ifndef SYS_GETANDROIDAPILEVEL_METHODDEF #define SYS_GETANDROIDAPILEVEL_METHODDEF #endif /* !defined(SYS_GETANDROIDAPILEVEL_METHODDEF) */ -/*[clinic end generated code: output=43b44240211afe95 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=5bda8e17c012884e input=a9049054013a1b77]*/ diff --git a/Python/sysmodule.c b/Python/sysmodule.c index 75e64553d88c9f..6c22f0527feaf4 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -2059,12 +2059,12 @@ sys.activate_stack_trampoline backend: str / -Activate the perf profiler trampoline. +Activate stack profiler trampoline *backend*. [clinic start generated code]*/ static PyObject * sys_activate_stack_trampoline_impl(PyObject *module, const char *backend) -/*[clinic end generated code: output=5783cdeb51874b43 input=b09020e3a17c78c5]*/ +/*[clinic end generated code: output=5783cdeb51874b43 input=a12df928758a82b4]*/ { #ifdef PY_HAVE_PERF_TRAMPOLINE if (strcmp(backend, "perf") == 0) { @@ -2095,12 +2095,12 @@ sys_activate_stack_trampoline_impl(PyObject *module, const char *backend) /*[clinic input] sys.deactivate_stack_trampoline -Dectivate the perf profiler trampoline. +Dectivate stack profiler trampoline backend. [clinic start generated code]*/ static PyObject * sys_deactivate_stack_trampoline_impl(PyObject *module) -/*[clinic end generated code: output=b50da25465df0ef1 input=491f4fc1ed615736]*/ +/*[clinic end generated code: output=b50da25465df0ef1 input=be0c9a8737fe7e8f]*/ { if (_PyPerfTrampoline_Init(0) < 0) { return NULL; @@ -2111,12 +2111,12 @@ sys_deactivate_stack_trampoline_impl(PyObject *module) /*[clinic input] sys.is_stack_trampoline_active -Returns *True* if the perf profiler trampoline is active. +Return *True* if the stack profiler trampoline is active. [clinic start generated code]*/ static PyObject * sys_is_stack_trampoline_active_impl(PyObject *module) -/*[clinic end generated code: output=ab2746de0ad9d293 input=061fa5776ac9dd59]*/ +/*[clinic end generated code: output=ab2746de0ad9d293 input=d802fd4a1afa2de8]*/ { #ifdef PY_HAVE_PERF_TRAMPOLINE if (_PyIsPerfTrampolineActive()) { From 574b39ab0fbbb936b31e134da7159dae3fc08ff9 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Wed, 31 Aug 2022 00:15:55 +0200 Subject: [PATCH 2/8] you => your --- Doc/howto/perf_profiling.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/perf_profiling.rst b/Doc/howto/perf_profiling.rst index d820d1701deb01..b4055eb35b89a5 100644 --- a/Doc/howto/perf_profiling.rst +++ b/Doc/howto/perf_profiling.rst @@ -195,7 +195,7 @@ information. This is because as the code that is interposed to allow ``perf`` support is dynamically generated it doesn't have any DWARF debugging information available. -You can check if you system has been compiled with this flag by running:: +You can check if your system has been compiled with this flag by running:: $ python -m sysconfig | grep 'no-omit-frame-pointer' From 9388964afbae9c8ffad30d5dc6087b3e81b46922 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Wed, 31 Aug 2022 00:17:16 +0200 Subject: [PATCH 3/8] Fixup What's New entry Co-authored-by: Pablo Galindo Salgado --- Doc/whatsnew/3.12.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index f8627164714b78..0ee88e793af43a 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -79,7 +79,7 @@ New Features :func:`sys.activate_stack_trampoline`, :func:`sys.deactivate_stack_trampoline`, and :func:`sys.is_stack_trampoline_active` APIs. - (Contributed by Pablo Galindo and Christian Heimes + (Design by Pablo Galindo. Contributed by Pablo Galindo and Christian Heimes with contributions from Gregory P. Smith [Google] and Mark Shannon in :gh:`96123`.) From 31c0eb819c2b951051e5cb12d497db01af28bcd4 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Wed, 31 Aug 2022 00:38:04 +0200 Subject: [PATCH 4/8] Address review: add links to perf wiki, and use bullet points --- Doc/howto/perf_profiling.rst | 19 +++++++++++-------- Doc/library/sys.rst | 3 ++- 2 files changed, 13 insertions(+), 9 deletions(-) diff --git a/Doc/howto/perf_profiling.rst b/Doc/howto/perf_profiling.rst index b4055eb35b89a5..a3c82b67198e47 100644 --- a/Doc/howto/perf_profiling.rst +++ b/Doc/howto/perf_profiling.rst @@ -8,10 +8,11 @@ Python support for the Linux ``perf`` profiler :author: Pablo Galindo -The Linux ``perf`` profiler is a very powerful tool that allows you to profile and -obtain information about the performance of your application. ``perf`` also has -a very vibrant ecosystem of tools that aid with the analysis of the data that it -produces. +`The Linux perf profiler `_ +is a very powerful tool that allows you to profile and obtain +information about the performance of your application. +``perf`` also has a very vibrant ecosystem of tools +that aid with the analysis of the data that it produces. The main problem with using the ``perf`` profiler with Python applications is that ``perf`` only allows to get information about native symbols, this is, the names of @@ -151,10 +152,12 @@ How to enable the ``perf`` profiling mode ----------------------------------------- There are three ways to activate the ``perf`` profiling mode: -using the :option:`-Xperf <-X>` command-line option, -using the :envvar:`PYTHONPERFSUPPORT` environment variable, -and using the :func:`sys.activate_stack_trampoline` and -:func:`sys.deactivate_stack_trampoline` APIs. + +* using the :option:`-Xperf <-X>` command-line option +* using the :envvar:`PYTHONPERFSUPPORT` environment variable +* using the :func:`sys.activate_stack_trampoline` and + :func:`sys.deactivate_stack_trampoline` APIs + If you want profiling to be active when you start the Python interpreter, use the :option:`-Xperf <-X>` option:: diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index ba982409a82dae..7fc1699a4a04c9 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1535,7 +1535,8 @@ always available. .. seealso:: - :ref:`perf_profiling` + * :ref:`perf_profiling` + * https://perf.wiki.kernel.org .. function:: deactivate_stack_trampoline() From 9c720233f0c61a748276efbac295702627edfdf3 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Wed, 31 Aug 2022 09:28:04 +0200 Subject: [PATCH 5/8] Address Victor's review --- Doc/howto/perf_profiling.rst | 12 ++++++------ Doc/using/cmdline.rst | 19 ++++++++----------- 2 files changed, 14 insertions(+), 17 deletions(-) diff --git a/Doc/howto/perf_profiling.rst b/Doc/howto/perf_profiling.rst index a3c82b67198e47..8ef9156e2d09b2 100644 --- a/Doc/howto/perf_profiling.rst +++ b/Doc/howto/perf_profiling.rst @@ -102,7 +102,7 @@ As you can see here, the Python functions are not shown in the output, only ``_P functions use the same C function to evaluate bytecode so we cannot know which Python function corresponds to which bytecode-evaluating function. -Instead, if we run the same experiment with ``perf`` support activated we get: +Instead, if we run the same experiment with ``perf`` support enabled we get: .. code-block:: shell-session @@ -148,10 +148,10 @@ Instead, if we run the same experiment with ``perf`` support activated we get: -How to enable the ``perf`` profiling mode ------------------------------------------ +How to enable ``perf`` profiling support +---------------------------------------- -There are three ways to activate the ``perf`` profiling mode: +There are three ways to enable ``perf`` profiling support: * using the :option:`-Xperf <-X>` command-line option * using the :envvar:`PYTHONPERFSUPPORT` environment variable @@ -163,11 +163,11 @@ use the :option:`-Xperf <-X>` option:: $ python -Xperf my_script.py -If you need to activate the ``perf`` profiling mode globally, +If you need to enable ``perf`` profiling support globally, set the environment variable :envvar:`PYTHONPERFSUPPORT` to a nonzero value. -If you need to dynamically activate and deactivate the ``perf`` profiling mode +If you need to dynamically enable and disable ``perf`` profiling in response to a signal or other communication mechanisms with your process, use the :func:`sys.activate_stack_trampoline` and :func:`sys.deactivate_stack_trampoline` APIs: diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst index e6c0e5cfa63cfa..5648048efaaca7 100644 --- a/Doc/using/cmdline.rst +++ b/Doc/using/cmdline.rst @@ -535,13 +535,11 @@ Miscellaneous options development (running from the source tree) then the default is "off". Note that the "importlib_bootstrap" and "importlib_bootstrap_external" frozen modules are always used, even if this flag is set to "off". - * ``-X perf`` activates compatibility mode with the ``perf`` profiler. - When this option is activated, the Linux ``perf`` profiler will be able to + * ``-X perf`` enables support for the Linux ``perf`` profiler. + When this option is provided, the ``perf`` profiler will be able to report Python calls. This option is only available on some platforms and will do nothing if is not supported on the current system. The default value - is "off". See :ref:`perf_profiling` for more details. - This option is equivalent to setting the environment variable - :envvar:`PYTHONPERFSUPPORT` to ``1``. + is "off". See also :envvar:`PYTHONPERFSUPPORT` and :ref:`perf_profiling`. It also allows passing arbitrary values and retrieving them through the :data:`sys._xoptions` dictionary. @@ -1036,14 +1034,13 @@ conflict. .. envvar:: PYTHONPERFSUPPORT - If this variable is set to a nonzero value, it activates compatibility mode - with the ``perf`` profiler so Python calls can be detected by it. - This is exactly equivalent to setting :option:`-X perf <-X>` - on the command line. + If this variable is set to a nonzero value, it enables support for + the Linux ``perf`` profiler so Python calls can be detected by it. - If set to ``0``, disable ``perf`` profiler support. + If set to ``0``, disable Linux ``perf`` profiler support. - See :ref:`perf_profiling` for more details. + See also the :option:`-X perf <-X>` command-line option + and :ref:`perf_profiling`. .. versionadded:: 3.12 From 3756252178722542747c879891ff3c9fd9e3177a Mon Sep 17 00:00:00 2001 From: "Gregory P. Smith" Date: Wed, 31 Aug 2022 15:59:42 -0700 Subject: [PATCH 6/8] https --- Doc/howto/perf_profiling.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/howto/perf_profiling.rst b/Doc/howto/perf_profiling.rst index 8ef9156e2d09b2..76f7c63856e8c4 100644 --- a/Doc/howto/perf_profiling.rst +++ b/Doc/howto/perf_profiling.rst @@ -8,7 +8,7 @@ Python support for the Linux ``perf`` profiler :author: Pablo Galindo -`The Linux perf profiler `_ +`The Linux perf profiler `_ is a very powerful tool that allows you to profile and obtain information about the performance of your application. ``perf`` also has a very vibrant ecosystem of tools From cbe8b162faacfa7d8a7d8be9a55b6cd52467bbd3 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Sun, 4 Sep 2022 22:59:50 +0200 Subject: [PATCH 7/8] Address reviews --- Doc/howto/perf_profiling.rst | 50 ++++++++++--------- Doc/library/sys.rst | 6 ++- Doc/whatsnew/3.12.rst | 7 +-- ...2-08-20-18-36-40.gh-issue-96143.nh3GFM.rst | 10 ++-- Python/clinic/sysmodule.c.h | 8 +-- Python/sysmodule.c | 10 ++-- 6 files changed, 52 insertions(+), 39 deletions(-) diff --git a/Doc/howto/perf_profiling.rst b/Doc/howto/perf_profiling.rst index 76f7c63856e8c4..f88a1dbe156c02 100644 --- a/Doc/howto/perf_profiling.rst +++ b/Doc/howto/perf_profiling.rst @@ -151,41 +151,45 @@ Instead, if we run the same experiment with ``perf`` support enabled we get: How to enable ``perf`` profiling support ---------------------------------------- -There are three ways to enable ``perf`` profiling support: +There are three ways to enable ``perf`` profiling support; +the environment variable :envvar:`PYTHONPERFSUPPORT` and the +:option:`-X perf <-X>` option allow you to enable perf profiling from the start, +whereas the :func:`sys.activate_stack_trampoline` and +:func:`sys.deactivate_stack_trampoline` functions allow you to enable and +disable it dynamically. -* using the :option:`-Xperf <-X>` command-line option -* using the :envvar:`PYTHONPERFSUPPORT` environment variable -* using the :func:`sys.activate_stack_trampoline` and - :func:`sys.deactivate_stack_trampoline` APIs +The :mod:`!sys` functions take precedence over the :option:`!-X` option, +the :option:`!-X` option takes precedence over the environment variable. -If you want profiling to be active when you start the Python interpreter, -use the :option:`-Xperf <-X>` option:: +Example, using the environment variable:: - $ python -Xperf my_script.py + $ PYTHONPERFSUPPORT=1 + $ python script1.py + $ python script2.py + $ python script3.py + $ perf report -g -i perf.data -If you need to enable ``perf`` profiling support globally, -set the environment variable :envvar:`PYTHONPERFSUPPORT` -to a nonzero value. +Example, using the :option:`!-X` option:: -If you need to dynamically enable and disable ``perf`` profiling -in response to a signal or other communication mechanisms with your process, -use the :func:`sys.activate_stack_trampoline` and -:func:`sys.deactivate_stack_trampoline` APIs: + $ python -X perf script1.py + $ perf report -g -i perf.data -.. code-block:: python +Example, using the :mod:`sys` APIs in file :file:`example.py`: - import sys - sys.activate_stack_trampoline("perf") +.. code-block:: python - # Run some code with Perf profiling active + import sys - sys.deactivate_stack_trampoline() + sys.activate_stack_trampoline("perf") + do_profiled_stuff() + sys.deactivate_stack_trampoline() - # Perf profiling is not active anymore + non_profiled_stuff() -Now we can analyze the data with ``perf report``:: +...then:: - $ perf report -g -i perf.data + $ python ./example.py + $ perf report -g -i perf.data How to obtain the best results diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 7fc1699a4a04c9..83c64b0e4b652b 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1540,7 +1540,9 @@ always available. .. function:: deactivate_stack_trampoline() - Deactivate the stack profiler trampoline. + Deactivate the current stack profiler trampoline backend. + + If no stack profiler is activated, this function has no effect. .. availability:: Linux. @@ -1548,7 +1550,7 @@ always available. .. function:: is_stack_trampoline_active() - Return ``True`` if the stack profiler trampoline is active. + Return ``True`` if a stack profiler trampoline is active. .. availability:: Linux. diff --git a/Doc/whatsnew/3.12.rst b/Doc/whatsnew/3.12.rst index 0ee88e793af43a..1be4b6cb737c73 100644 --- a/Doc/whatsnew/3.12.rst +++ b/Doc/whatsnew/3.12.rst @@ -74,9 +74,10 @@ Important deprecations, removals or restrictions: New Features ============ -* Add :ref:`perf_profiling` through the new command-line option - :option:`-X perf <-X>`, as well as the new - :func:`sys.activate_stack_trampoline`, +* Add :ref:`perf_profiling` through the new + environment variable :envvar:`PYTHONPERFSUPPORT`, + the new command-line option :option:`-X perf <-X>`, + as well as the new :func:`sys.activate_stack_trampoline`, :func:`sys.deactivate_stack_trampoline`, and :func:`sys.is_stack_trampoline_active` APIs. (Design by Pablo Galindo. Contributed by Pablo Galindo and Christian Heimes diff --git a/Misc/NEWS.d/next/Core and Builtins/2022-08-20-18-36-40.gh-issue-96143.nh3GFM.rst b/Misc/NEWS.d/next/Core and Builtins/2022-08-20-18-36-40.gh-issue-96143.nh3GFM.rst index 377816bc31eab9..ac31e3b858c126 100644 --- a/Misc/NEWS.d/next/Core and Builtins/2022-08-20-18-36-40.gh-issue-96143.nh3GFM.rst +++ b/Misc/NEWS.d/next/Core and Builtins/2022-08-20-18-36-40.gh-issue-96143.nh3GFM.rst @@ -1,6 +1,8 @@ -Add :ref:`perf_profiling` through the new command-line option -:option:`-X perf <-X>`, as well as the new -:func:`sys.activate_stack_trampoline`, :func:`sys.deactivate_stack_trampoline`, +Add :ref:`perf_profiling` through the new +environment variable :envvar:`PYTHONPERFSUPPORT`, +the new command-line option :option:`-X perf <-X>`, +as well as the new :func:`sys.activate_stack_trampoline`, +:func:`sys.deactivate_stack_trampoline`, and :func:`sys.is_stack_trampoline_active` APIs. -Design by Pablo Galindo. Patch by Pablo Galindo and Christian Heimes +Design by Pablo Galindo. Contributed by Pablo Galindo and Christian Heimes with contributions from Gregory P. Smith [Google] and Mark Shannon. diff --git a/Python/clinic/sysmodule.c.h b/Python/clinic/sysmodule.c.h index b455c83e43f728..36dcf71dd6fe49 100644 --- a/Python/clinic/sysmodule.c.h +++ b/Python/clinic/sysmodule.c.h @@ -1192,7 +1192,9 @@ PyDoc_STRVAR(sys_deactivate_stack_trampoline__doc__, "deactivate_stack_trampoline($module, /)\n" "--\n" "\n" -"Dectivate stack profiler trampoline backend."); +"Deactivate the current stack profiler trampoline backend.\n" +"\n" +"If no stack profiler is activated, this function has no effect."); #define SYS_DEACTIVATE_STACK_TRAMPOLINE_METHODDEF \ {"deactivate_stack_trampoline", (PyCFunction)sys_deactivate_stack_trampoline, METH_NOARGS, sys_deactivate_stack_trampoline__doc__}, @@ -1210,7 +1212,7 @@ PyDoc_STRVAR(sys_is_stack_trampoline_active__doc__, "is_stack_trampoline_active($module, /)\n" "--\n" "\n" -"Return *True* if the stack profiler trampoline is active."); +"Return *True* if a stack profiler trampoline is active."); #define SYS_IS_STACK_TRAMPOLINE_ACTIVE_METHODDEF \ {"is_stack_trampoline_active", (PyCFunction)sys_is_stack_trampoline_active, METH_NOARGS, sys_is_stack_trampoline_active__doc__}, @@ -1267,4 +1269,4 @@ sys_is_stack_trampoline_active(PyObject *module, PyObject *Py_UNUSED(ignored)) #ifndef SYS_GETANDROIDAPILEVEL_METHODDEF #define SYS_GETANDROIDAPILEVEL_METHODDEF #endif /* !defined(SYS_GETANDROIDAPILEVEL_METHODDEF) */ -/*[clinic end generated code: output=5bda8e17c012884e input=a9049054013a1b77]*/ +/*[clinic end generated code: output=5c9e637db409f41e input=a9049054013a1b77]*/ diff --git a/Python/sysmodule.c b/Python/sysmodule.c index 6c22f0527feaf4..7ec4ba1bc34954 100644 --- a/Python/sysmodule.c +++ b/Python/sysmodule.c @@ -2095,12 +2095,14 @@ sys_activate_stack_trampoline_impl(PyObject *module, const char *backend) /*[clinic input] sys.deactivate_stack_trampoline -Dectivate stack profiler trampoline backend. +Deactivate the current stack profiler trampoline backend. + +If no stack profiler is activated, this function has no effect. [clinic start generated code]*/ static PyObject * sys_deactivate_stack_trampoline_impl(PyObject *module) -/*[clinic end generated code: output=b50da25465df0ef1 input=be0c9a8737fe7e8f]*/ +/*[clinic end generated code: output=b50da25465df0ef1 input=9f629a6be9fe7fc8]*/ { if (_PyPerfTrampoline_Init(0) < 0) { return NULL; @@ -2111,12 +2113,12 @@ sys_deactivate_stack_trampoline_impl(PyObject *module) /*[clinic input] sys.is_stack_trampoline_active -Return *True* if the stack profiler trampoline is active. +Return *True* if a stack profiler trampoline is active. [clinic start generated code]*/ static PyObject * sys_is_stack_trampoline_active_impl(PyObject *module) -/*[clinic end generated code: output=ab2746de0ad9d293 input=d802fd4a1afa2de8]*/ +/*[clinic end generated code: output=ab2746de0ad9d293 input=29616b7bf6a0b703]*/ { #ifdef PY_HAVE_PERF_TRAMPOLINE if (_PyIsPerfTrampolineActive()) { From db4aaa219f7e1090a44ec6a85f4c929eaf125c59 Mon Sep 17 00:00:00 2001 From: "Erlend E. Aasland" Date: Tue, 4 Oct 2022 12:27:19 +0200 Subject: [PATCH 8/8] Apply suggestions from code review Co-authored-by: Ezio Melotti --- Doc/howto/perf_profiling.rst | 11 +++++------ Doc/library/sys.rst | 2 +- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/Doc/howto/perf_profiling.rst b/Doc/howto/perf_profiling.rst index f88a1dbe156c02..f02ce3a077b1ef 100644 --- a/Doc/howto/perf_profiling.rst +++ b/Doc/howto/perf_profiling.rst @@ -151,12 +151,11 @@ Instead, if we run the same experiment with ``perf`` support enabled we get: How to enable ``perf`` profiling support ---------------------------------------- -There are three ways to enable ``perf`` profiling support; -the environment variable :envvar:`PYTHONPERFSUPPORT` and the -:option:`-X perf <-X>` option allow you to enable perf profiling from the start, -whereas the :func:`sys.activate_stack_trampoline` and -:func:`sys.deactivate_stack_trampoline` functions allow you to enable and -disable it dynamically. +``perf`` profiling support can either be enabled from the start using +the environment variable :envvar:`PYTHONPERFSUPPORT` or the +:option:`-X perf <-X>` option, +or dynamically using :func:`sys.activate_stack_trampoline` and +:func:`sys.deactivate_stack_trampoline`. The :mod:`!sys` functions take precedence over the :option:`!-X` option, the :option:`!-X` option takes precedence over the environment variable. diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 76b04027ebade1..36f90d9b1a7790 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1558,7 +1558,7 @@ always available. .. function:: activate_stack_trampoline(backend, /) Activate the stack profiler trampoline *backend*. - The only supported backend is `"perf"`. + The only supported backend is ``"perf"``. .. availability:: Linux.