Skip to content

Commit 8b40505

Browse files
henryiiidean0x7d
henryiii
authored and
dean0x7d
committed
Utility for redirecting C++ streams to Python (#1009)
1 parent 3d8df5a commit 8b40505

File tree

8 files changed

+560
-3
lines changed

8 files changed

+560
-3
lines changed

‎docs/advanced/pycpp/utilities.rst

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,72 @@ expected in Python:
2121
auto args = py::make_tuple("unpacked", true);
2222
py::print("->", *args, "end"_a="<-"); // -> unpacked True <-
2323
24+
.. _ostream_redirect:
25+
26+
Capturing standard output from ostream
27+
======================================
28+
29+
Often, a library will use the streams ``std::cout`` and ``std::cerr`` to print,
30+
but this does not play well with Python's standard ``sys.stdout`` and ``sys.stderr``
31+
redirection. Replacing a library's printing with `py::print <print>` may not
32+
be feasible. This can be fixed using a guard around the library function that
33+
redirects output to the corresponding Python streams:
34+
35+
.. code-block:: cpp
36+
37+
#include <pybind11/iostream.h>
38+
39+
...
40+
41+
// Add a scoped redirect for your noisy code
42+
m.def("noisy_func", []() {
43+
py::scoped_ostream_redirect stream(
44+
std::cout, // std::ostream&
45+
py::module::import("sys").attr("stdout") // Python output
46+
);
47+
call_noisy_func();
48+
});
49+
50+
This method respects flushes on the output streams and will flush if needed
51+
when the scoped guard is destroyed. This allows the output to be redirected in
52+
real time, such as to a Jupyter notebook. The two arguments, the C++ stream and
53+
the Python output, are optional, and default to standard output if not given. An
54+
extra type, `py::scoped_estream_redirect <scoped_estream_redirect>`, is identical
55+
except for defaulting to ``std::cerr`` and ``sys.stderr``; this can be useful with
56+
`py::call_guard`, which allows multiple items, but uses the default constructor:
57+
58+
.. code-block:: py
59+
60+
// Alternative: Call single function using call guard
61+
m.def("noisy_func", &call_noisy_function,
62+
py::call_guard<py::scoped_ostream_redirect,
63+
py::scoped_estream_redirect>());
64+
65+
The redirection can also be done in Python with the addition of a context
66+
manager, using the `py::add_ostream_redirect() <add_ostream_redirect>` function:
67+
68+
.. code-block:: cpp
69+
70+
py::add_ostream_redirect(m, "ostream_redirect");
71+
72+
The name in Python defaults to ``ostream_redirect`` if no name is passed. This
73+
creates the following context manager in Python:
74+
75+
.. code-block:: python
76+
77+
with ostream_redirect(stdout=True, stderr=True):
78+
noisy_function()
79+
80+
It defaults to redirecting both streams, though you can use the keyword
81+
arguments to disable one of the streams if needed.
82+
83+
.. note::
84+
85+
The above methods will not redirect C-level output to file descriptors, such
86+
as ``fprintf``. For those cases, you'll need to redirect the file
87+
descriptors either directly in C or with Python's ``os.dup2`` function
88+
in an operating-system dependent way.
89+
2490
.. _eval:
2591

2692
Evaluating Python expressions from strings and files

‎docs/changelog.rst

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -123,10 +123,15 @@ v2.2.0 (Not yet released)
123123
7. Fixed lifetime of temporary C++ objects created in Python-to-C++ conversions.
124124
`#924 <https://github.com/pybind/pybind11/pull/924>`_.
125125

126-
* Scope guard call policy for RAII types, e.g. ``py::call_guard<py::gil_scoped_release>()``.
127-
See :ref:`call_policies` for details.
126+
* Scope guard call policy for RAII types, e.g. ``py::call_guard<py::gil_scoped_release>()``,
127+
``py::call_guard<py::scoped_ostream_redirect>()``. See :ref:`call_policies` for details.
128128
`#740 <https://github.com/pybind/pybind11/pull/740>`_.
129129

130+
* Utility for redirecting C++ streams to Python (e.g. ``std::cout`` ->
131+
``sys.stdout``). Scope guard ``py::scoped_ostream_redirect`` in C++ and
132+
a context manager in Python. See :ref:`ostream_redirect`.
133+
`#1009 <https://github.com/pybind/pybind11/pull/1009>`_.
134+
130135
* Improved handling of types and exceptions across module boundaries.
131136
`#915 <https://github.com/pybind/pybind11/pull/915>`_,
132137
`#951 <https://github.com/pybind/pybind11/pull/951>`_,
@@ -298,7 +303,6 @@ v2.2.0 (Not yet released)
298303
`#923 <https://github.com/pybind/pybind11/pull/923>`_,
299304
`#963 <https://github.com/pybind/pybind11/pull/963>`_.
300305

301-
302306
v2.1.1 (April 7, 2017)
303307
-----------------------------------------------------
304308

‎docs/reference.rst

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,15 @@ Embedding the interpreter
7171

7272
.. doxygenclass:: scoped_interpreter
7373

74+
Redirecting C++ streams
75+
=======================
76+
77+
.. doxygenclass:: scoped_ostream_redirect
78+
79+
.. doxygenclass:: scoped_estream_redirect
80+
81+
.. doxygenfunction:: add_ostream_redirect
82+
7483
Python build-in functions
7584
=========================
7685

‎include/pybind11/iostream.h

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
/*
2+
pybind11/iostream.h -- Tools to assist with redirecting cout and cerr to Python
3+
4+
Copyright (c) 2017 Henry F. Schreiner
5+
6+
All rights reserved. Use of this source code is governed by a
7+
BSD-style license that can be found in the LICENSE file.
8+
*/
9+
10+
#pragma once
11+
12+
#include "pybind11.h"
13+
14+
#include <streambuf>
15+
#include <ostream>
16+
#include <string>
17+
#include <memory>
18+
#include <iostream>
19+
20+
NAMESPACE_BEGIN(PYBIND11_NAMESPACE)
21+
NAMESPACE_BEGIN(detail)
22+
23+
// Buffer that writes to Python instead of C++
24+
class pythonbuf : public std::streambuf {
25+
private:
26+
using traits_type = std::streambuf::traits_type;
27+
28+
char d_buffer[1024];
29+
object pywrite;
30+
object pyflush;
31+
32+
int overflow(int c) {
33+
if (!traits_type::eq_int_type(c, traits_type::eof())) {
34+
*pptr() = traits_type::to_char_type(c);
35+
pbump(1);
36+
}
37+
return sync() ? traits_type::not_eof(c) : traits_type::eof();
38+
}
39+
40+
int sync() {
41+
if (pbase() != pptr()) {
42+
// This subtraction cannot be negative, so dropping the sign
43+
str line(pbase(), static_cast<size_t>(pptr() - pbase()));
44+
45+
pywrite(line);
46+
pyflush();
47+
48+
setp(pbase(), epptr());
49+
}
50+
return 0;
51+
}
52+
53+
public:
54+
pythonbuf(object pyostream)
55+
: pywrite(pyostream.attr("write")),
56+
pyflush(pyostream.attr("flush")) {
57+
setp(d_buffer, d_buffer + sizeof(d_buffer) - 1);
58+
}
59+
60+
/// Sync before destroy
61+
~pythonbuf() {
62+
sync();
63+
}
64+
};
65+
66+
NAMESPACE_END(detail)
67+
68+
69+
/** \rst
70+
This a move-only guard that redirects output.
71+
72+
.. code-block:: cpp
73+
74+
#include <pybind11/iostream.h>
75+
76+
...
77+
78+
{
79+
py::scoped_ostream_redirect output;
80+
std::cout << "Hello, World!"; // Python stdout
81+
} // <-- return std::cout to normal
82+
83+
You can explicitly pass the c++ stream and the python object,
84+
for example to guard stderr instead.
85+
86+
.. code-block:: cpp
87+
88+
{
89+
py::scoped_ostream_redirect output{std::cerr, py::module::import("sys").attr("stderr")};
90+
std::cerr << "Hello, World!";
91+
}
92+
\endrst */
93+
class scoped_ostream_redirect {
94+
protected:
95+
std::streambuf *old;
96+
std::ostream &costream;
97+
detail::pythonbuf buffer;
98+
99+
public:
100+
scoped_ostream_redirect(
101+
std::ostream &costream = std::cout,
102+
object pyostream = module::import("sys").attr("stdout"))
103+
: costream(costream), buffer(pyostream) {
104+
old = costream.rdbuf(&buffer);
105+
}
106+
107+
~scoped_ostream_redirect() {
108+
costream.rdbuf(old);
109+
}
110+
111+
scoped_ostream_redirect(const scoped_ostream_redirect &) = delete;
112+
scoped_ostream_redirect(scoped_ostream_redirect &&other) = default;
113+
scoped_ostream_redirect &operator=(const scoped_ostream_redirect &) = delete;
114+
scoped_ostream_redirect &operator=(scoped_ostream_redirect &&) = delete;
115+
};
116+
117+
118+
/** \rst
119+
Like `scoped_ostream_redirect`, but redirects cerr by default. This class
120+
is provided primary to make ``py::call_guard`` easier to make.
121+
122+
.. code-block:: cpp
123+
124+
m.def("noisy_func", &noisy_func,
125+
py::call_guard<scoped_ostream_redirect,
126+
scoped_estream_redirect>());
127+
128+
\endrst */
129+
class scoped_estream_redirect : public scoped_ostream_redirect {
130+
public:
131+
scoped_estream_redirect(
132+
std::ostream &costream = std::cerr,
133+
object pyostream = module::import("sys").attr("stderr"))
134+
: scoped_ostream_redirect(costream,pyostream) {}
135+
};
136+
137+
138+
NAMESPACE_BEGIN(detail)
139+
140+
// Class to redirect output as a context manager. C++ backend.
141+
class OstreamRedirect {
142+
bool do_stdout_;
143+
bool do_stderr_;
144+
std::unique_ptr<scoped_ostream_redirect> redirect_stdout;
145+
std::unique_ptr<scoped_estream_redirect> redirect_stderr;
146+
147+
public:
148+
OstreamRedirect(bool do_stdout = true, bool do_stderr = true)
149+
: do_stdout_(do_stdout), do_stderr_(do_stderr) {}
150+
151+
void enter() {
152+
if (do_stdout_)
153+
redirect_stdout.reset(new scoped_ostream_redirect());
154+
if (do_stderr_)
155+
redirect_stderr.reset(new scoped_estream_redirect());
156+
}
157+
158+
void exit() {
159+
redirect_stdout.reset();
160+
redirect_stderr.reset();
161+
}
162+
};
163+
164+
NAMESPACE_END(detail)
165+
166+
/** \rst
167+
This is a helper function to add a C++ redirect context manager to Python
168+
instead of using a C++ guard. To use it, add the following to your binding code:
169+
170+
.. code-block:: cpp
171+
172+
#include <pybind11/iostream.h>
173+
174+
...
175+
176+
py::add_ostream_redirect(m, "ostream_redirect");
177+
178+
You now have a Python context manager that redirects your output:
179+
180+
.. code-block:: python
181+
182+
with m.ostream_redirect():
183+
m.print_to_cout_function()
184+
185+
This manager can optionally be told which streams to operate on:
186+
187+
.. code-block:: python
188+
189+
with m.ostream_redirect(stdout=true, stderr=true):
190+
m.noisy_function_with_error_printing()
191+
192+
\endrst */
193+
inline class_<detail::OstreamRedirect> add_ostream_redirect(module m, std::string name = "ostream_redirect") {
194+
return class_<detail::OstreamRedirect>(m, name.c_str(), module_local())
195+
.def(init<bool,bool>(), arg("stdout")=true, arg("stderr")=true)
196+
.def("__enter__", &detail::OstreamRedirect::enter)
197+
.def("__exit__", [](detail::OstreamRedirect &self, args) { self.exit(); });
198+
}
199+
200+
NAMESPACE_END(PYBIND11_NAMESPACE)

‎setup.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@
2828
'include/pybind11/embed.h',
2929
'include/pybind11/eval.h',
3030
'include/pybind11/functional.h',
31+
'include/pybind11/iostream.h',
3132
'include/pybind11/numpy.h',
3233
'include/pybind11/operators.h',
3334
'include/pybind11/options.h',

‎tests/CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ set(PYBIND11_TEST_FILES
4040
test_eval.cpp
4141
test_exceptions.cpp
4242
test_factory_constructors.cpp
43+
test_iostream.cpp
4344
test_kwargs_and_defaults.cpp
4445
test_local_bindings.cpp
4546
test_methods_and_attributes.cpp

0 commit comments

Comments
 (0)