NEW: MouseEvents for InputSystem - Sending data from package & Samples and Tests #2207
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅ @@ Coverage Diff @@
## develop #2207 +/- ##
===========================================
+ Coverage 68.14% 76.71% +8.57%
===========================================
Files 367 465 +98
Lines 53685 87919 +34234
===========================================
+ Hits 36584 67448 +30864
- Misses 17101 20471 +3370 Flags with carried forward coverage won't be shown. Click here to find out more.
... and 100 files with indirect coverage changes 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Nice work @ritamerkl ! I like the approach and the sample is of reasonable size to test the main functionality. Also really nice to have plenty of tests.
However:
- This functionality needs to be guarded with defines in asmdef, as this will only be available from 6.3 forward AFAIK. So if I try this PR without the trunk PR, I'll get compilation errors.
- I would like to see at least 1 test with Touch as well, particularly with more than 1 touch. Mouse and Pen are pretty similar as they are a single pointer.
Also there's some CI failures that will need to be addressed. Not sure if it's only because of the lack of defines or if there's any impact on pointer devices functionality. |
|
WIP: working on adding touch samples (multi-touch) & parallel pointers (eg. touch&pen). |
untitled folder/Il2CppOutputProject/Source/il2cppOutput/il2cpp-compile.traceevents
Outdated
Show resolved
Hide resolved
ritamerkl
changed the title
| InvokeAfterUpdateCallback(updateType); | ||
| //send pointer data to backend for OnMouseEvents | ||
| if (Pointer.current != null && gameIsPlaying) | ||
| NativeInputSystem.SetMouseEventsData(Pointer.current.press.isPressed, Pointer.current.press.wasPressedThisFrame, Pointer.current.position.x.value, Pointer.current.position.y.value); |
There was a problem hiding this comment.
Shouldn't this be routed via INativeInputRuntime interface and its NativeInputRuntime and TestInputRuntime implementations?
There was a problem hiding this comment.
NativeInputRuntime is the name of the Bindings class in the module, I fail to understand what exactly you are referring to.
I placed it here to have all updated data after the inputsystem Update, this happens for test as well.
There was a problem hiding this comment.
I referred to the fact that this is calling NativeInputSystem directly. If you look at NativeInputRuntime.cs you will see it implements INativeInputRuntime as an abstraction of the module to allow stubbing it away in e.g. tests. Hence InputTestRuntime also exists stubbing these calls. It's basically a mocking strategy that has historically been used. If this is possible to auto-test without mocking, it should right? Then I see no reason why this would be just fine, but just wanted to let you know in case you had not noticed since it seems SetMouseEventsData is part of Input module based on where its placed/mapped?
There was a problem hiding this comment.
And NativeInputRuntime is not the bindings class, it's just an abstraction within the package. The bindings class (P/Invoke) is called NativeInputSystem which is the one you are calling here
There was a problem hiding this comment.
I was adding this same comment. We should call this as m_Runtime.SetMouseEventsData(...);. This avoids making other classes (like this one) dependent on the bindings as well. This way we only have NativeInputRuntime class as a single dependent of the Module bindings (NativeInputSystem) and make it consistent across the codebase as is.
And by making it part of IInputRuntime interface we leverage the mocking strategy @ekcoh mentioned as well. So I think this should be improved.
There was a problem hiding this comment.
Alright, so I understand what you are after. I confused the names :D
I was considering a change until I noticed:
- including SetMouseEventsData into the IInputRuntime would mean we would need a implementation for InputTestRuntime as well, since there is no way to mock setting the data, but we really need to send the data to the module to achieve reliable test results, this would mean we have to include NativeInputSystem there (so there are no savings on bindings).
- there is no way to mock sending the data for mouse events, so it does not really make sense to bloat the code with two extra methods in NativeInputRuntime/InputTestRuntime.
I would leave this as is, I guess that is fine for this kind. feel free to object if I missed something.
There was a problem hiding this comment.
Hm, good point. Yeah, not sure how we could mock this in InputTestRuntime then. And from what you mentioned, it doesn't seem to make sense to have it in InputTestRuntime as well, if we really need to call the module.
I don't have a better answer right now so maybe this is fine as is.
It just feels odd to be breaking the class contract that in NativeInputRuntime should be the only file referencing NativeInputSystem. But maybe that's ok in this case?
Anyway, this is not a blocker at all. I'll leave it approved.
…echnologies/InputSystem into mouseevents-samles-tests
| using UnityEngine.InputSystem.EnhancedTouch; | ||
| using Touch = UnityEngine.InputSystem.EnhancedTouch.Touch; | ||
|
|
||
| public class OnMouseEventSample : MonoBehaviour |
There was a problem hiding this comment.
If the intent here is to serve as a sample (and maybe also our own way to manually test or debug this), I would suggest that we have all the supported implicit script mouse event callback functions present in this file.
There was a problem hiding this comment.
E.g. missing OnMouseOver, OnMouseEnter, OnMouseExit, OnMouseUp, OnMouseUpAsButton
There was a problem hiding this comment.
Right, added here: 0bef5d2
| @@ -0,0 +1,199 @@ | |||
| #if UNITY_6000_4_OR_NEWER | |||
There was a problem hiding this comment.
Great you added automated tests for all the events! This way we protect ourselves from regressions in this area. Similarly I think there is a lot to gain from having them covered in the sample as well.
|
|
||
| private static HashSet<Type> _testDevices = new HashSet<Type> { typeof(Mouse), typeof(Pen), typeof(Touchscreen) }; | ||
| // OnMouseOver/Exit and Hover events are not supported for touch | ||
| private static HashSet<Type> _testDevicesNoTouch = new HashSet<Type> { typeof(Mouse), typeof(Pen) }; |
There was a problem hiding this comment.
Isn't a HashSet overkill here (dynamic allocation etc) when function with a switch statement with 2 cases would be enough?
There was a problem hiding this comment.
I can't see (or am I missing something) that we use lookup in the tests? If not, then I would recommend a plain static readonly array for these. (Less allocation, leaner, simpler) and also likely much faster. Also, when using HashSet, the iteration order is "random"
| [Category("MouseEvents")] | ||
| public IEnumerator MouseEvents_CanReceiveOnMouseDown([ValueSource(nameof(_testDevices))] Type pointerType) | ||
| { | ||
| var pointer = (Pointer)InputSystem.s_Manager.AddDevice(pointerType); |
There was a problem hiding this comment.
Nitpick: It looks like this
var pointer = (Pointer)InputSystem.s_Manager.AddDevice(pointerType);
InputSystem.AddDevice(pointer);Is repeated for all the tests. Can't we just embed it into SetUpScene() and have it as an out parameter from that function to reduce redundant code?
|
|
||
| ### Added | ||
| - Exposed MediaPlayPause, MediaRewind, MediaForward keys on Keyboard. | ||
| - Added OnMouse events for the InputSystem from editor version 6.3, including samples and tests. |
There was a problem hiding this comment.
I would suggest saying "Added support for (MonoBehavior OnMouse-events) [https://docs.unity3d.com/ScriptReference/MonoBehaviour.html] when running the Input System on Unity 6.3 or newer."
Then I would make a separate entry for the sample linking to it. It makes more sense to say Unity 6.3 IMO since editor might indicate it would only be supported in editor but this is for players as well right?
There was a problem hiding this comment.
good idea, especially the link. Changed here: a9cff76
| //send pointer data to backend for OnMouseEvents | ||
| #if UNITY_6000_4_OR_NEWER | ||
| if (Pointer.current != null && gameIsPlaying) | ||
| NativeInputSystem.SetMouseEventsData(Pointer.current.press.isPressed, Pointer.current.press.wasPressedThisFrame, Pointer.current.position.x.value, Pointer.current.position.y.value); |
There was a problem hiding this comment.
Thanks for putting this together @ritamerkl, I added a few suggestions and my only main concern is still whether this will behave similar to Input Manager when we only call into the SetMouseEventData(...) function once every frame - this looks incorrect to me. I would have expected that this would not be based on a polling-based evaluation like this here but rather sit inside the event processing loop above or from within the device state updates after event compression. Does this work for e.g. 1 FPS?
Maybe I am overlooking something.... but was Input Manager only calling this once every frame?
There was a problem hiding this comment.
I think we talked about this before, but to clarify: before this was happening on a PlayerLoopCallback, which was set on module registration. The player loop happens once a frame, the timing of this is "PreUpdate".
Unless I am terribly mistaking something here, I think that is the same logic, except the Input update is set to FixedUpdate or custom (and changed to any other time).
|
I see CI is failing with "Packages\com.unity.inputsystem\InputSystem\InputManager.cs(3710,35): error CS0117: 'NativeInputSystem' does not contain a definition for 'SetMouseEventsData'" but this would be expected until the feature is available in Unity. |
It is weird since this is wrapped in defines for unity 600.4 or newer, but still fails on 600.3 🤔 |
| InvokeAfterUpdateCallback(updateType); | ||
| //send pointer data to backend for OnMouseEvents | ||
| #if UNITY_6000_4_OR_NEWER | ||
| if (Pointer.current != null && gameIsPlaying) |
There was a problem hiding this comment.
This means we will be calling this every frame, regardless of having received an event from a Pointer device or not, right?
There was a problem hiding this comment.
We need to look at how its implemented under the hood, there might even be timeout logic in there which would require it to be once a frame.
There was a problem hiding this comment.
there might even be timeout logic in there which would require it to be once a frame.
I'm not sure I understand what you mean with this. Could you specify it a bit?
There was a problem hiding this comment.
There is no timeout logic for the MouseEvents.cs if you mean that
| //// same goes for events that someone may queue from a change monitor callback | ||
| InvokeAfterUpdateCallback(updateType); | ||
| //send pointer data to backend for OnMouseEvents | ||
| #if UNITY_6000_4_OR_NEWER |
There was a problem hiding this comment.
I would advise against this. Instead, create an asmdef symbol that defines the feature/capability, e.g. UNITY_INPUTSYSTEM_HAS_MOUSE_SCRIPT_EVENTS_SUPPORTED, then let that symbol be conditionally compiled based on the specific version that contains the native changes, e.g. 6000.0.4.a7, that is more specific and also removes versions from the source code which only cares about a certain API being a certain way instead. And it centralises the version number mess. At least this is what we agreed upon in #2176, and the same technique was used in #2200 to handle the API difference in module API and behaviour .
There was a problem hiding this comment.
Right, very good point. I did that here: 7b13b3e
| InvokeAfterUpdateCallback(updateType); | ||
| //send pointer data to backend for OnMouseEvents | ||
| if (Pointer.current != null && gameIsPlaying) | ||
| NativeInputSystem.SetMouseEventsData(Pointer.current.press.isPressed, Pointer.current.press.wasPressedThisFrame, Pointer.current.position.x.value, Pointer.current.position.y.value); |
There was a problem hiding this comment.
Hm, good point. Yeah, not sure how we could mock this in InputTestRuntime then. And from what you mentioned, it doesn't seem to make sense to have it in InputTestRuntime as well, if we really need to call the module.
I don't have a better answer right now so maybe this is fine as is.
It just feels odd to be breaking the class contract that in NativeInputRuntime should be the only file referencing NativeInputSystem. But maybe that's ok in this case?
Anyway, this is not a blocker at all. I'll leave it approved.
|
Without your trunk PR I get: Packages\com.unity.inputsystem\InputSystem\InputManager.cs(3711,35): error CS0117: 'NativeInputSystem' does not contain a definition for 'DoSendMouseEvents' This needs a version check? Also, should this really be a shipped sample? It seems more like a simple test scene (not to knock down its value, it is just maybe too niche?) |
Description
This PR introduces sending pointer data (e.g. mouse, pen, touch...) to native for the OnMouse events driven by the InputSystem. It also adds tests and samples for the OnMouse events working with the InputSystem. The behaviour on native side is introduced with this PR and both PR's should land together, or one (shortly) after the other.
Testing status & QA
Only manual testing for the sample.
Overall Product Risks
Comments to reviewers
Please review together with https://github.cds.internal.unity3d.com/unity/unity/pull/73198.
Checklist
Before review:
Changed,Fixed,Addedsections.Area_CanDoX,Area_CanDoX_EvenIfYIsTheCase,Area_WhenIDoX_AndYHappens_ThisIsTheResult.During merge:
NEW: ___.FIX: ___.DOCS: ___.CHANGE: ___.RELEASE: 1.1.0-preview.3.After merge: