reflect: implement the unique reflect rfc #7207

soqb · 2023-01-15T10:53:57Z

Objective

Implements the Unique Reflect RFC.

Solution

Implements the RFC.
This implementation differs in some ways from the RFC:
- In the RFC, it was suggested Reflect: Any but PartialReflect: ?Any. During initial implementation I tried this, but we assume the PartialReflect: 'static in a lot of places and the changes required crept out of the scope of this PR.
- PartialReflect::try_into_reflect originally returned Option<Box<dyn Reflect>> but i changed this to Result<Box<dyn Reflect>, Box<dyn PartialReflect>> since the method takes by value and otherwise there would be no way to recover the type. as_full and as_full_mut both still return Option<&(mut) dyn Reflect>.

Changelog

Added PartialReflect.
Reflect is now a subtrait of PartialReflect.
Moved most methods on Reflect to the new PartialReflect.
Added PartialReflect::{as_partial_reflect, as_partial_reflect_mut, into_partial_reflect}.
Added PartialReflect::{try_as_reflect, try_as_reflect_mut, try_into_reflect}.
Added <dyn PartialReflect>::{try_downcast_ref, try_downcast_mut, try_downcast, try_take} supplementing the methods on dyn Reflect.

Migration Guide

Most instances of dyn Reflect should be changed to dyn PartialReflect which is less restrictive, however trait bounds should generally stay as T: Reflect.
The new PartialReflect::{as_partial_reflect, as_partial_reflect_mut, into_partial_reflect, try_as_reflect, try_as_reflect_mut, try_into_reflect} methods as well as Reflect::{as_reflect, as_reflect_mut, into_reflect} will need to be implemented for manual implementors of Reflect.

Future Work

This PR is designed to be followed up by another "Unique Reflect Phase 2" that addresses the following points:
- Investigate making serialization revolve around Reflect instead of PartialReflect.
- Remove the try_* methods on dyn PartialReflect since they are stop gaps.
- Investigate usages like ReflectComponent. In the places they currently use PartialReflect, should they be changed to use Reflect?
Merging this opens the door to lots of reflection features we haven't been able to implement.
- We could re-add the Reflectable trait and make FromReflect a requirement to improve FromReflect ergonomics. This is currently not possible because dynamic types cannot sensibly be FromReflect.
- Since this is an alternative to bevy_reflect: Add Reflectable trait #5772, bevy_reflect: Recursive registration #5781 would be made cleaner.

nicopap · 2023-01-15T11:58:46Z

Yooo! Awesome. Thank you so much for taking the time to implement the RFC. I'm sorry for not doing it myself. This would unlock the next step for bevy_reflect. I'll at least take the time to review this thoroughfully.

nicopap

Theory hits reality at full speed and hard!

The PR is actually not bad, it's just a lot of simple changes. So I encourage people to check it out. It's just a bit time consuming.

This really questions the necessity/feasibility of unique reflect. With this PR, the problems the RFC sets to solve are not solved. And I'm not sure there is a clear path to actually solve them. Specifically, how could we get rid of the try_downcast_ref() methods on dyn PartialReflect while staying ergonomic?

I really don't want to force users to do my_value.as_full().unwrap().downcast_ref() instead of my_value.try_downcast_ref() everywhere.

I think a bit I missed in the RFC is this:

Most things should now accept a dyn PartialReflect rather than
dyn Reflect.

Should be:

Most things should now accept a dyn PartialReflect rather than
dyn Reflect.

Most thing that return dyn Reflect should still return dyn Reflect.

Then unique reflect would make sense, otherwise it's just needless boilerplate.

Is it an achievable goal? If not, I think we should drop the unique Reflect RFC.

If it is achievable (even if not within this PR), then just a few comments and this is good to go.

crates/bevy_asset/src/reflect.rs

crates/bevy_input/src/gamepad.rs

crates/bevy_reflect/src/utility.rs

crates/bevy_reflect/src/reflect.rs

crates/bevy_reflect/src/lib.rs

crates/bevy_reflect/src/impls/std.rs

crates/bevy_reflect/src/reflect.rs

soqb · 2023-01-21T17:38:56Z

Is it an achievable goal? If not, I think we should drop the unique Reflect RFC.

i think it's probably alright. most of the diff in this PR come from just renaming Reflect to PartialReflect before anything else was changed so it was all a bit hasty.

returning dyn Reflect over dyn PartialReflect where applicable definitely feels like a good practice.

i'll give the changes a once over when i get the time because as you've pointed out a lot of return types and probably some trait bounds are PartialReflect when they should really be Reflect.

i reckon some of the changes will be better suited to a follow-up PR merged in the same release (i'm not expecting this to be merged by 0.10) and/or when the current situation surrounding reflect_hash/reflect_partial_eq/serializable is resolved but i'll do what i think is reasonable for this PR.

nicopap

Looks good. Just some minor issues to address.

crates/bevy_asset/src/reflect.rs

examples/reflection/reflection.rs

crates/bevy_reflect/src/impls/std.rs

crates/bevy_reflect/src/reflect.rs

crates/bevy_reflect/src/impls/std.rs

crates/bevy_reflect/src/reflect.rs

crates/bevy_reflect/src/impls/std.rs

crates/bevy_asset/src/assets.rs

crates/bevy_reflect/bevy_reflect_derive/src/impls/structs.rs

crates/bevy_reflect/bevy_reflect_derive/src/impls/full_reflect.rs

MrGVSV · 2023-09-11T00:26:51Z

crates/bevy_reflect/src/reflect.rs

-    fn into_any(self: Box<Self>) -> Box<dyn Any>;
+    /// Returns the value as a fully-reflected [`&dyn Reflect`](Reflect),
+    /// or [`None`] if the value does not implement it.
+    fn try_as_reflect(&self) -> Option<&dyn Reflect>;


Bikeshed: I think the convention I've seen is that try_*** methods return a Result. I wonder if we should do the same. That might mean adding a new error type for the dynamic types to return.

We could also go back to naming these as_reflect and just return the Option.

Same for the methods on dyn PartialReflect.

The RFC uses "canonical" for Reflect, so it's probably better to keep to .as_canonical or even .canonical_ref() , considering std::Any has a downcast_ref returning Option<&T>.

"Canonical" risks getting users asking "does this method only works on ubuntu-based linux distributions". But it is probably easier as an implementor to keep consistent, since that's the terminology we settled on.

I think the convention I've seen is that try_*** methods return a Result.

this is certainly the most common form for try_*** methods, but i don't see any reason why Options shouldn't be included. the Try trait is implemented for both types, and so are supported by ?. std's methods try_for_each etc. support Option.

even if we consensus disagrees i would prefer returning a Result<&dyn Reflect, BikeshedUnitReflectUpcastError> than changing the name of the method.

i don't prefer as_canonical at all. it's both vague and conceptual: my rule of thumb for good naming is that users should be able to understand why the method has its name from just its signature and documentation, without doing any wider research and i don't think a method like this should have more than two lines of documentation.

even if we consensus disagrees i would prefer returning a Result<&dyn Reflect, BikeshedUnitReflectUpcastError> than changing the name of the method.

Yeah I agree. It might also set us up for potential error handling in the future.

i don't prefer as_canonical at all. it's both vague and conceptual

Yeah canonical works for the RFC, but in practice I think I like the reflect/partial_reflect relation better. I think it will be a lot easier to grasp for newcomers.

The objections make sense. I'm not married to canonical. I think "try_as_foo" is a bit weird though. Maybe "as_reflect" or "try_reflect"

MrGVSV · 2023-09-11T00:38:39Z

crates/bevy_reflect/src/map.rs

-        key: Box<dyn Reflect>,
-        mut value: Box<dyn Reflect>,
-    ) -> Option<Box<dyn Reflect>> {
+        key: Box<dyn PartialReflect>,


Most dynamic types don't implement reflect_hash. So maybe until #8695, we should enforce this by making the key dyn Reflect (and same for all other methods that require a hashable key). While this would be more strict, it would also be more helpful since we could catch this error at compile time rather than runtime.

Thoughts?

i'd rather not change this method twice. reflect_hash is quite goofy but i'd rather not induce two sets of churn for an often-unused api with a solution already in the works.

I wouldn't say it's unused— we get a fair number of people inadvertently running into this. But your reasoning makes sense and we can just wait for #8695 to land before slapping on bandaids.

nicopap

I approve, because it makes a much needed change and I don't see anything wrong. But I really think we should reconsider naming.

I looked back at the RFC. And I think the old name was better. This adds a lot of noise to the Reflect API. Basically 70% of the code change here could be removed if we swapped what was renamed.

It's important to have the most common type be the short name, so that when naming things, the additional qualifier in the name shows what's special about it. It builds on how we human pattern-match the world. The other way around is confusing IMO.

So the RFC used CanonReflect. What about:

CanonicalReflect or UniqueReflect for what the RFC calls Reflect
Reflect for what the RFC calls PartialReflect

nicopap · 2023-09-11T10:45:27Z

crates/bevy_ecs/src/reflect/component.rs

    pub contains: fn(EntityRef) -> bool,
    /// Function pointer implementing [`ReflectComponent::reflect()`].
-    pub reflect: fn(EntityRef) -> Option<&dyn Reflect>,
+    pub reflect: fn(EntityRef) -> Option<&dyn PartialReflect>,


The components are always stored as concrete types, might as well return the restricted type.

Suggested change

pub reflect: fn(EntityRef) -> Option<&dyn PartialReflect>,

pub reflect: fn(EntityRef) -> Option<&dyn Reflect>,

i totally agree that this can be changed without much resistance but i'm not sure we want to. currently all ComponentIds map 1-to-1 with Rust types, but if we (and we do want to) support dynamic component types this relationship will change, and i think there's good reasons to want to reflect those components with this api.

I still think we need a place where we collect all the "either PartialReflect or Reflect" decisions. If you are willing to write a comment in the RFC discussions, I'll take care to update the RFC with such a new section.

nicopap · 2023-09-11T10:45:37Z

crates/bevy_ecs/src/reflect/component.rs

+    pub reflect: fn(EntityRef) -> Option<&dyn PartialReflect>,
    /// Function pointer implementing [`ReflectComponent::reflect_mut()`].
-    pub reflect_mut: for<'a> fn(&'a mut EntityMut<'_>) -> Option<Mut<'a, dyn Reflect>>,
+    pub reflect_mut: for<'a> fn(&'a mut EntityMut<'_>) -> Option<Mut<'a, dyn PartialReflect>>,


same here

Suggested change

pub reflect_mut: for<'a> fn(&'a mut EntityMut<'_>) -> Option<Mut<'a, dyn PartialReflect>>,

pub reflect_mut: for<'a> fn(&'a mut EntityMut<'_>) -> Option<Mut<'a, dyn Reflect>>,

nicopap · 2023-09-11T10:47:12Z

crates/bevy_ecs/src/reflect/resource.rs

    pub remove: fn(&mut World),
    /// Function pointer implementing [`ReflectResource::reflect()`].
-    pub reflect: fn(&World) -> Option<&dyn Reflect>,
+    pub reflect: fn(&World) -> Option<&dyn PartialReflect>,


Suggested change

pub reflect: fn(&World) -> Option<&dyn PartialReflect>,

pub reflect: fn(&World) -> Option<&dyn Reflect>,

nicopap · 2023-09-11T10:54:51Z

crates/bevy_reflect/src/reflect.rs

-    fn into_any(self: Box<Self>) -> Box<dyn Any>;
+    /// Returns the value as a fully-reflected [`&dyn Reflect`](Reflect),
+    /// or [`None`] if the value does not implement it.
+    fn try_as_reflect(&self) -> Option<&dyn Reflect>;


The RFC uses "canonical" for Reflect, so it's probably better to keep to .as_canonical or even .canonical_ref() , considering std::Any has a downcast_ref returning Option<&T>.

"Canonical" risks getting users asking "does this method only works on ubuntu-based linux distributions". But it is probably easier as an implementor to keep consistent, since that's the terminology we settled on.

nicopap · 2023-09-11T11:15:03Z

crates/bevy_reflect/src/tuple_struct.rs

 /// struct MyTupleStruct(usize);
 ///
-/// let my_tuple_struct: &dyn Reflect = &MyTupleStruct(123);
+/// let my_tuple_struct: &dyn PartialReflect = &MyTupleStruct(123);


Suggested change

/// let my_tuple_struct: &dyn PartialReflect = &MyTupleStruct(123);

/// let my_tuple_struct: &dyn Reflect = &MyTupleStruct(123);

crates/bevy_reflect/src/type_info.rs

crates/bevy_reflect/bevy_reflect_derive/src/impls/enums.rs

alice-i-cecile · 2023-09-29T15:31:19Z

@soqb, are you able to clean this up? This is in the 0.12 milestone and I'd like to move this forward, but it looks like there's more work to be done here.

soqb · 2023-10-02T07:17:25Z

sorry for the late reply, i've had an incredibly busy week but hope to find some time tonight to address what i need to here.

MrGVSV · 2023-10-02T07:45:54Z

I approve, because it makes a much needed change and I don't see anything wrong. But I really think we should reconsider naming.

I looked back at the RFC. And I think the old name was better. This adds a lot of noise to the Reflect API. Basically 70% of the code change here could be removed if we swapped what was renamed.

It's important to have the most common type be the short name, so that when naming things, the additional qualifier in the name shows what's special about it. It builds on how we human pattern-match the world. The other way around is confusing IMO.

So the RFC used CanonReflect. What about:

CanonicalReflect or UniqueReflect for what the RFC calls Reflect

Reflect for what the RFC calls PartialReflect

This was briefly talked about in this thread, but I'd like to push back on this. If I remember right, the reason we chose Reflect and PartialReflect is that it allowed newcomers to associate them to similarly-named traits (e.g. Eq and PartialEq). This association helps demonstrate that Reflect is a stricter subset of PartialReflect, and that PartialReflect is "good enough" for most use cases, again similar to traits like Eq and PartialEq.

However, I will admit that CanonicalReflect or UniqueReflect might better indicate where that strictness comes from (i.e. that it's the actual, canonical type it claims it is). I'm still not a huge fan of the canonical terminology, though. And if we did go with something like this, I feel we may also want to consider naming Reflect to something like DynamicReflect to really indicate that it's a "looser" version of CanonicalReflect.

So my vote is for PartialReflect/Reflect, but that's just one person's opinion so feel free to disagree with me haha

nicopap · 2023-10-02T08:50:08Z

Pointing that out to avoid regrets down the line. Unlike PartialOrd or PartialEq, PartialReflect is meant to be used as trait objects, and it is present all over the place.

Instead of seeing PartialReflect as a partial version of Reflect, we can see Reflect as being a specialized PartialReflect. Rust has Iterator and ExactSizeIterator for example, not PartialIterator and Iterator. So it would make sense to have a Reflect and Un1kReflect (silly name used for illustration).

I'm fine with the current compromise though.

MrGVSV · 2023-10-02T19:20:05Z

Pointing that out to avoid regrets down the line. Unlike PartialOrd or PartialEq, PartialReflect is meant to be used as trait objects, and it is present all over the place.

I don't think its usage as a trait object prevents us from using the Partial naming scheme. People use PartialEq as bounds and such a lot (and more than they do for Eq), which I think plays well into how we want people to prefer PartialReflect unless they truly need a full Reflect value.

Instead of seeing PartialReflect as a partial version of Reflect, we can see Reflect as being a specialized PartialReflect. Rust has Iterator and ExactSizeIterator for example, not PartialIterator and Iterator. So it would make sense to have a Reflect and Un1kReflect (silly name used for illustration).

I think the difference with Iterator comes from the fact that there's a lot of different Iterator traits in the Rust standard library. None of them could really be a "PartialIterator" since they all have different meanings for what an "iterator" is. We really only have two reflection traits: the stricter one and the looser one.

That being said, I see your point about not needing a DynamicReflect like I suggested. Iterator isn't called BasicIterator or anything like that, so we don't need to do the same for Reflect if we end up using something like CanonicalReflect/UniqueReflect.

I'm fine with the current compromise though.

I left a message on Discord to draw more attention to this issue since the entire PR is a big shift and we want to ensure the community is generally on board with whatever we end up doing.

soqb · 2023-10-03T15:16:23Z

I'm fine with the current compromise though.

i think it boils down to what is the most accessible. even with good docs, UniqueReflect or even {Canonical, Full, Serializing}Reflect are non-obvious names where people are familiar with PartialFoo, if they see it, they'll know very roughly what the name represents.

to me, it seems more helpful to say PartialReflect is Reflect without some of the important bits, compared to UniqueReflect is Reflect with extra important bits. under either naming scheme, people will reach for Reflect first when they are unsure, and under the first scheme, this will more often be adequate.

crates/bevy_scene/src/dynamic_scene.rs

alice-i-cecile

Looks solid: no blocking concerns! I've merged a couple of grammar nits. @soqb, once this is merge conflict free ping me and let's get this merged.

…t-unique-reflect

Co-authored-by: Alice Cecile <[email protected]> Co-authored-by: Gino Valente <[email protected]>

alice-i-cecile · 2024-08-12T16:24:58Z

A doc link on ReflectSet needs cleaning up before this can merge: https://github.com/bevyengine/bevy/actions/runs/10324580350/job/28584326492?pr=7207#step:6:2042

alice-i-cecile · 2024-10-20T14:26:13Z

Thank you to everyone involved with the authoring or reviewing of this PR! This work is relatively important and needs release notes! Head over to bevyengine/bevy-website#1671 if you'd like to help out.

Shatur · 2024-11-06T21:11:47Z

Looks like this PR missing in the migration guide for some reason.

nicopap self-assigned this Jan 15, 2023

alice-i-cecile added A-Reflection Runtime information about types C-Feature A new feature, making something new possible X-Controversial There is active debate or serious implications around merging this PR labels Jan 15, 2023

soqb marked this pull request as ready for review January 16, 2023 14:21

nicopap mentioned this pull request Jan 18, 2023

bevy_reflect: Unique Reflect bevyengine/rfcs#56

Merged

5 tasks

nicopap reviewed Jan 18, 2023

View reviewed changes

nicopap self-requested a review January 23, 2023 08:12

soqb force-pushed the reflect-unique-reflect branch from 8e3488c to 910f654 Compare January 28, 2023 12:51

nicopap suggested changes Jan 30, 2023

View reviewed changes

MrGVSV self-requested a review January 31, 2023 22:39

soqb force-pushed the reflect-unique-reflect branch from 9f10455 to 6663dfc Compare February 5, 2023 16:47

MrGVSV reviewed Feb 6, 2023

View reviewed changes

nicopap self-requested a review August 20, 2023 18:47

nicopap removed their assignment Aug 26, 2023

alice-i-cecile added this to the 0.12 milestone Aug 29, 2023

MrGVSV reviewed Sep 11, 2023

View reviewed changes

nicopap approved these changes Sep 11, 2023

View reviewed changes

alice-i-cecile requested a review from MrGVSV September 29, 2023 15:29

nicopap reviewed Oct 4, 2023

View reviewed changes

crates/bevy_scene/src/dynamic_scene.rs Outdated Show resolved Hide resolved

alice-i-cecile modified the milestones: 0.12, 0.13 Oct 20, 2023

alice-i-cecile approved these changes Aug 8, 2024

View reviewed changes

alice-i-cecile added S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Aug 8, 2024

soqb and others added 3 commits August 9, 2024 19:27

Merge branch 'main' of https://github.com/bevyengine/bevy into reflec…

38f482b

…t-unique-reflect

use impl_full_reflect in more places

30f4009

Grammar nits

6363a74

Co-authored-by: Alice Cecile <[email protected]> Co-authored-by: Gino Valente <[email protected]>

soqb force-pushed the reflect-unique-reflect branch from ea99863 to 6363a74 Compare August 9, 2024 18:46

soqb added 3 commits August 9, 2024 19:59

function registry fixes

b872913

ci fixes part i/xxiv

cb0c7ac

compile fail :((((((((((

d80c1b4

alice-i-cecile added S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it and removed S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged labels Aug 9, 2024

alice-i-cecile enabled auto-merge August 12, 2024 15:33

alice-i-cecile disabled auto-merge August 12, 2024 16:49

alice-i-cecile added 2 commits August 12, 2024 12:49

Merge branch 'main' into reflect-unique-reflect

0ab6079

Fix doc link for ReflectSet

5354ecb

alice-i-cecile enabled auto-merge August 12, 2024 16:52

alice-i-cecile added this pull request to the merge queue Aug 12, 2024

Merged via the queue into bevyengine:main with commit 6ab8767 Aug 12, 2024

MrGVSV mentioned this pull request Sep 9, 2024

bevy_reflect: Add DynamicTyped trait #15108

Merged

alice-i-cecile mentioned this pull request Oct 20, 2024

Write release notes for PR #7207: reflect: implement the unique reflect rfc bevyengine/bevy-website#1671

Closed

	pub reflect: fn(EntityRef) -> Option<&dyn PartialReflect>,
	pub reflect: fn(EntityRef) -> Option<&dyn Reflect>,

	pub reflect_mut: for<'a> fn(&'a mut EntityMut<'_>) -> Option<Mut<'a, dyn PartialReflect>>,
	pub reflect_mut: for<'a> fn(&'a mut EntityMut<'_>) -> Option<Mut<'a, dyn Reflect>>,

	pub reflect: fn(&World) -> Option<&dyn PartialReflect>,
	pub reflect: fn(&World) -> Option<&dyn Reflect>,

	/// let my_tuple_struct: &dyn PartialReflect = &MyTupleStruct(123);
	/// let my_tuple_struct: &dyn Reflect = &MyTupleStruct(123);

Uh oh!

reflect: implement the unique reflect rfc #7207

reflect: implement the unique reflect rfc #7207

Uh oh!

Conversation

soqb commented Jan 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Objective

Solution

Changelog

Migration Guide

Future Work

Uh oh!

nicopap commented Jan 15, 2023

Uh oh!

nicopap left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

soqb commented Jan 21, 2023

Uh oh!

nicopap left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nicopap left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

soqb commented Jan 15, 2023 •

edited

Loading

soqb commented Oct 3, 2023 •

edited

Loading