Skip to content

Commit bf1b8e4

Browse files
committed
Improve documentation of Python and C++ exceptions
The main change is to treat error_already_set as a separate category of exception that arises in different circumstances and needs to be handled differently. The asymmetry between Python and C++ exceptions is further emphasized.
1 parent 4d9024e commit bf1b8e4

File tree

3 files changed

+125
-29
lines changed

3 files changed

+125
-29
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,3 +38,4 @@ sosize-*.txt
3838
pybind11Config*.cmake
3939
pybind11Targets.cmake
4040
/*env*
41+
/.vscode

docs/advanced/exceptions.rst

Lines changed: 114 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,19 @@
11
Exceptions
22
##########
33

4-
Built-in exception translation
5-
==============================
4+
Built-in C++ to Python exception translation
5+
============================================
6+
7+
When Python calls C++ code through pybind11, pybind11 provides a C++ exception handler
8+
that will trap C++ exceptions, translate them to the corresponding Python exception,
9+
and raise them so that Python code can handle them.
610

7-
When C++ code invoked from Python throws an ``std::exception``, it is
8-
automatically converted into a Python ``Exception``. pybind11 defines multiple
9-
special exception classes that will map to different types of Python
10-
exceptions:
11+
pybind11 defines translations for ``std::exception`` and its standard
12+
subclasses, and several special exception classes that translate to specific
13+
Python exceptions. Note that these are not actually Python exceptions, so they
14+
cannot be examined using the Python C API. Instead, they are pure C++ objects
15+
that pybind11 will translate the corresponding Python exception when they arrive
16+
at its exception handler.
1117

1218
.. tabularcolumns:: |p{0.5\textwidth}|p{0.45\textwidth}|
1319

@@ -46,22 +52,11 @@ exceptions:
4652
| | ``__setitem__`` in dict-like |
4753
| | objects, etc.) |
4854
+--------------------------------------+--------------------------------------+
49-
| :class:`pybind11::error_already_set` | Indicates that the Python exception |
50-
| | flag has already been set via Python |
51-
| | API calls from C++ code; this C++ |
52-
| | exception is used to propagate such |
53-
| | a Python exception back to Python. |
54-
+--------------------------------------+--------------------------------------+
5555

56-
When a Python function invoked from C++ throws an exception, pybind11 will convert
57-
it into a C++ exception of type :class:`error_already_set` whose string payload
58-
contains a textual summary. If you call the Python C-API directly, and it
59-
returns an error, you should ``throw py::error_already_set();``, which allows
60-
pybind11 to deal with the exception and pass it back to the Python interpreter.
61-
(Another option is to call ``PyErr_Clear`` in the
62-
`Python C-API <https://docs.python.org/3/c-api/exceptions.html#c.PyErr_Clear>`_
63-
to clear the error. The Python error must be thrown or cleared, or Python/pybind11
64-
will be left in an invalid state.)
56+
Exception translation is not bidirectional. That is, *catching* the C++
57+
exceptions defined above above will not trap exceptions that originate from
58+
Python. For that, catch :class:`pybind11::error_already_set`. See :ref:`below
59+
<handling_python_exceptions_cpp>` for further details.
6560

6661
There is also a special exception :class:`cast_error` that is thrown by
6762
:func:`handle::call` when the input arguments cannot be converted to Python
@@ -106,7 +101,6 @@ and use this in the associated exception translator (note: it is often useful
106101
to make this a static declaration when using it inside a lambda expression
107102
without requiring capturing).
108103

109-
110104
The following example demonstrates this for a hypothetical exception classes
111105
``MyCustomException`` and ``OtherException``: the first is translated to a
112106
custom python exception ``MyCustomError``, while the second is translated to a
@@ -149,20 +143,111 @@ section.
149143
may be explicitly (re-)thrown to delegate it to the other,
150144
previously-declared existing exception translators.
151145

146+
.. _handling_python_exceptions_cpp:
147+
148+
Handling exceptions from Python in C++
149+
======================================
150+
151+
When C++ calls Python functions, such as in a callback function or when
152+
manipulating Python objects, and Python raises an ``Exception``, pybind11
153+
converts the Python exception into a C++ exception of type
154+
:class:`pybind11::error_already_set` whose payload contains a C++ string textual
155+
summary and the actual Python exception. ``error_already_set`` is used to
156+
propagate Python exception back to Python (or possibly, handle them in C++).
157+
158+
For example:
159+
160+
.. code-block:: cpp
161+
162+
try {
163+
// open("missing.txt", "r")
164+
auto file = py::module::import("io").attr("open")("missing.txt", "r");
165+
auto text = file.attr("read")();
166+
file.attr("close")();
167+
} catch (py::error_already_set &e) {
168+
if (e.matches(PyExc_FileNotFoundError)) {
169+
py::print("missing.txt not found");
170+
} else if (e.match(PyExc_PermissionError)) {
171+
py::print("missing.txt found but not accessible");
172+
} else {
173+
throw;
174+
}
175+
}
176+
177+
Note that C++ to Python exception translation does not apply here, since that is
178+
a method for translating C++ exceptions to Python, not vice versa. The error raised
179+
from Python is always ``error_already_set``.
180+
181+
This example illustrates this behavior:
182+
183+
.. code-block:: cpp
184+
185+
try {
186+
py::eval("raise ValueError('The Ring')");
187+
} catch (py::value_error &boromir) {
188+
// Boromir never gets the ring
189+
assert(false);
190+
} catch (py::error_already_set &frodo) {
191+
// Frodo gets the ring
192+
py::print("I will take the ring");
193+
}
194+
195+
try {
196+
// py::value_error is a request for pybind11 to raise a Python exception
197+
throw py::value_error("The ball");
198+
} catch (py::error_already_set &cat) {
199+
// cat won't catch the ball since
200+
// py::value_error is not a Python exception
201+
assert(false);
202+
} catch (py::value_error &dog) {
203+
// dog will catch the ball
204+
py::print("Run Spot run");
205+
throw; // Throw it again (pybind11 will raise ValueError)
206+
}
207+
208+
Handling errors from the Python C API
209+
=====================================
210+
211+
Where possible, use :ref:`pybind11 wrappers <wrappers>` instead of calling
212+
the Python C API it directly. If you do call the Python C API directly, in
213+
addition to manually managing reference counts, you must follow the pybind11
214+
error protocol, which is outlined here.
215+
216+
After calling the Python C API, if Python returns an error, you should
217+
``throw py::error_already_set();``, which allows pybind11 to deal with the
218+
exception and pass it back to the Python interpreter. This includes calls to
219+
the error setting functions such as ``PyErr_SetString``.
220+
221+
.. code-block:: cpp
222+
223+
PyErr_SetString(PyExc_TypeError, "C API type error demo");
224+
throw py::error_already_set();
225+
226+
// But it would be easier to simply...
227+
throw py::type_error("pybind11 wrapper type error");
228+
229+
Alternately, you call `PyErr_Clear
230+
<https://docs.python.org/3/c-api/exceptions.html#c.PyErr_Clear>`_ to clear the
231+
error.
232+
233+
Any Python error must be thrown or cleared, or Python/pybind11 will be left in
234+
an invalid state.
235+
152236
.. _unraisable_exceptions:
153237

154238
Handling unraisable exceptions
155239
==============================
156240

157241
If a Python function invoked from a C++ destructor or any function marked
158242
``noexcept(true)`` (collectively, "noexcept functions") throws an exception, there
159-
is no way to propagate the exception, as such functions may not throw at
160-
run-time.
161-
162-
Neither Python nor C++ allow exceptions raised in a noexcept function to propagate. In
163-
Python, an exception raised in a class's ``__del__`` method is logged as an
164-
unraisable error. In Python 3.8+, a system hook is triggered and an auditing
165-
event is logged. In C++, ``std::terminate()`` is called to abort immediately.
243+
is no way to propagate the exception, as such functions may not throw.
244+
``std::terminate()`` is called to abort immediately.
245+
246+
Similarly, Python exceptions raised in a class's ``__del__`` method do not
247+
propagate, but are logged by Python as an unraisable error. In Python 3.8+, a
248+
`system hook is triggered
249+
<https://docs.python.org/3/library/sys.html#sys.unraisablehook>`_
250+
and an auditing event is logged.
166251

167252
Any noexcept function should have a try-catch block that traps
168253
class:`error_already_set` (or any other exception that can occur). Note that pybind11

docs/advanced/pycpp/object.rst

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
Python types
22
############
33

4+
.. _wrappers:
5+
46
Available wrappers
57
==================
68

@@ -168,3 +170,11 @@ Generalized unpacking according to PEP448_ is also supported:
168170
Python functions from C++, including keywords arguments and unpacking.
169171

170172
.. _PEP448: https://www.python.org/dev/peps/pep-0448/
173+
174+
Handling exceptions
175+
===================
176+
177+
Python exceptions from wrapper classes will be thrown as a ``py::error_already_set``.
178+
See :ref:`Handling exceptions from Python in C++
179+
<handling_python_exceptions_cpp>` for more information on handling exceptions
180+
raised when calling C++ wrapper classes.

0 commit comments

Comments
 (0)