Skip to content

Conversation

jagerman
Copy link
Member

This reimplements the std::reference_wrapper<T> caster to be a shell around the underlying T caster (rather than assuming T is a generic type), which lets it work for things like std::reference_wrapper<int> or anything else custom type caster with a lvalue cast operator.

Fixes #848: this also makes it properly fail when None is provided, just as an ordinary lvalue reference argument would similarly fail.

This also adds a static assert to test that T has an appropriate type caster. It triggers for casters like std::pair, which have return-by-value cast operators. (In theory this could be supported by storing a local temporary for such types, but that's beyond the scope of this PR). This shouldn't break anything in terms of backwards compatibility: currently reference_wrapper only actually works for generic types, which always have an lvalue cast operator.

@jagerman
Copy link
Member Author

Struck by random CI failures on both appveyor and travis on this one. (Restarted the travis, and the appveyor is just a single #792 failure).

static handle cast(const std::reference_wrapper<type> &src, return_value_policy policy, handle parent) {
return type_caster_base<type>::cast(&src.get(), policy, parent);
return caster_t::cast(&src.get(), policy, parent);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This existed before this PR, but I'm just noticing: casting as a pointer here means that rvp::automatic will behave like rvp::take_ownership which goes completely against std::reference_wrapper. Would there be anything wrong with just hard coding rvp::reference here? I don't think any other policy could possibly be valid.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable to me.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I agree that rvp::automatic and take_ownership seem wrong and should be masked, but copy could be valid. Perhaps it would be better to just set it to rvp::reference if it's set to automatic or take_ownership, and otherwise leave it alone?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When is copy valid?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy is the default currently if you py::cast(x) a reference_wrapper<T> x. Essentially both copy and reference currently work the way I'd expect:

#include <pybind11/pybind11.h>

namespace py = pybind11;

struct Foo { int value = 2; };

std::vector<std::reference_wrapper<Foo>> foos() {
    static Foo foo1, foo2;
    std::vector<std::reference_wrapper<Foo>> r;
    r.emplace_back(foo1);
    r.emplace_back(foo2);
    return r;
}

PYBIND11_PLUGIN(copy) {
    py::module m("copy");

    py::class_<Foo>(m, "Foo").def(py::init<>()).def_readwrite("value", &Foo::value);

    m.def("z1", []() {
        py::list l;
        for (auto &f : foos()) l.append(py::cast(f, py::return_value_policy::copy));
        return l;
    });
    m.def("z2", []() {
        py::list l;
        for (auto &f : foos()) l.append(py::cast(f, py::return_value_policy::reference));
        return l;
    });

    return m.ptr();
}
>>> from copy import z1, z2, Foo
>>> a = z1()
>>> a2 = z1()
>>> b = z2()
>>> b2 = z2()
>>> a[0] is a2[0]
False
>>> b[0] is b2[0]
True

(The ::copy there is redundant: it'll be the default for an lvalue reference wrapper already.)

Copy link
Member Author

@jagerman jagerman May 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though now that I think about it, I'm not sure why we pass a pointer here at all, instead of just passing an lvalue reference (i.e. changing &src.get() -> src.get()).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't &src.get() -> src.get() make the following a copy?

T x = ...; // T is a user-defined type
py::function f = ...;
f(std::ref(x)); // <-- rvp::automatic_reference: the `cast(const&)` overload does a copy 

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That doesn't actually copy (because of the rvalue), but if you change the reference wrapper argument to an lvalue it is indeed a problem.

I'll change back to the pointer cast, with a take_ownership mask.

@jagerman jagerman force-pushed the refwrapper-fixes branch 2 times, most recently from aedd297 to eb8a19c Compare May 27, 2017 03:34
@dean0x7d
Copy link
Member

Looks like there's a conflict now. But otherwise, looks good to me.

jagerman added 2 commits May 28, 2017 21:54
test_issues is deprecated, and the following commit adds other, related
reference_wrapper tests.
This reimplements the std::reference_wrapper<T> caster to be a shell
around the underlying T caster (rather than assuming T is a generic
type), which lets it work for things like `std::reference_wrapper<int>`
or anything else custom type caster with a lvalue cast operator.

This also makes it properly fail when None is provided, just as an
ordinary lvalue reference argument would similarly fail.

This also adds a static assert to test that T has an appropriate type
caster.  It triggers for casters like `std::pair`, which have
return-by-value cast operators.  (In theory this could be supported by
storing a local temporary for such types, but that's beyond the scope
of this PR).

This also replaces `automatic` or `take_ownership` return value policies
with `automatic_reference` as taking ownership of a reference inside a
reference_wrapper is not valid.
@jagerman jagerman force-pushed the refwrapper-fixes branch from eb8a19c to 216c9d1 Compare May 29, 2017 02:25
@jagerman jagerman merged commit acedd6c into pybind:master May 30, 2017
@dean0x7d dean0x7d modified the milestone: v2.2 Aug 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants