Element click behavior on touchscreen devices

[ogarai]

This is a follow-up from the Chrome Driver issue https://issues.chromium.org/issues/42320346.

The element click spec currently assumes a mouse interaction. However, it is desirable for element click to produce touch events (touchstart, touchend) when webdriver tests are run on touchscreen mobile devices, as a real user would use a tap gesture there, and many mobile sites are listening only for touch events.

Some corner cases need to be considered here:

• A desktop device with a touchscreen could mean there is a mouse as well as a touchscreen, in which case generating touch events by default may not be desirable.
• When emulating a mobile touchscreen device on desktop, it may need to behave similar to a real touchscreen-only device and generate touch events there as well.

It would be good to get clarification on whether touch events should be generated when performing an element click and how to ensure it is done where only touch and no mouse interactions are expected.

Alternatively, should there be a separate element tap command instead?

[OrKoN]

Can you achieve the desired outcome by calling https://w3c.github.io/webdriver/#perform-actions with pointer actions https://w3c.github.io/webdriver/#pointer-input-source?

[ogarai]

Can you achieve the desired outcome by calling https://w3c.github.io/webdriver/#perform-actions with pointer actions https://w3c.github.io/webdriver/#pointer-input-source?

Yes, I suppose so. I believe the original motivation for the chromedriver issue was that on touchscreen-only devices element.click() producing mouse events does not closely resemble a real user interaction on those devices. So would it be make sense to address that to some extent, whether it's a documentation change, or producing touch events from element.click() on touch-only devices, or providing a distinct element interaction command for tap?

[OrKoN]

Sorry, what I meant is that if the ChromeDriver correctly emits touch events if you the perform-actions API. Reading the click spec it seems like in the step 8 the mouse events are only dispatched for option elements and otherwise it should fallback to the pointer events that should result in touch events being emitted. Perhaps on the implementation side in Chromium https://chromedevtools.github.io/devtools-protocol/tot/Emulation/#method-setEmitTouchEventsForMouse could be used if it's not done already.

[OrKoN]

It would be also good to add WPT tests for this scenario to make sure it's handled consistently across platforms and browsers https://github.com/web-platform-tests/wpt/tree/master/webdriver/tests/classic/element_click

[ogarai]

otherwise it should fallback to the pointer events that should result in touch events being emitted

Right, it doesn't specify the pointer source though, and chrome seems to pick mouse as default on both desktop and mobile, not sure about other browsers.

So there is still a bit of a gap in determining when to emit touch events, i.e. always when a touchscreen is present regardless of desktop/mobile or only on mobile touchscreen devices, and which of the following makes most sense to address this:

1. Explicitly mentioned in the spec how to determine that
2. Up to the browsers to chose automatically
3. Up to the user to specify (e.g. something similar to chromedriver's Emulation.setEmitTouchEventsForMouse)

[gsnedders]

Unless I'm mistaken, with Safari on iOS a touch event is emitted — and that feels very much like the correct behaviour. Having it behave like a mouse click, which is something that can only occur on iOS with Assistive Touch enabled in the Accessibility settings and a Bluetooth mouse connected, isn't really helpful when it comes to developer expectations of testing what users would actually do.

I think we probably want to make it implementation defined, but I would expect it to broadly follow from the value of the pointer media feature.

[ogarai]

I think we probably want to make it implementation defined, but I would expect it to broadly follow from the value of the pointer media feature.

This is a great suggestion. If I'm not mistaken, typically when both mouse and touchscreen inputs are present, a mouse is chosen as the primary pointing device because it is more precise. And so it could be assumed that implementations would generate touch events when there is a touchscreen and no mouse, and not when both are present.

In this case I wonder if the spec should be updated to include a note about the interpretation of pointer sources being implementation defined, and expected to match the interpretation of primary pointing device as per the pointer media feature.

Also, if the recommendation is for it to be implementation defined, I suppose WPT tests for this wouldn't be applicable any more?

[gsnedders]

Our options are, I think:

1. Require it to always be "mouse"
2. Require it to always be "touch"
3. Require it to be "mouse" if the pointer media feature is equal to fine and require it to be "touch" if the pointer media feature is equal to coarse`
4. Recommend it to always be "mouse"
5. Recommend it to always be "touch"
6. Recommend it to be "mouse" if the pointer media feature is equal to fine and recommend it to be "touch" if the pointer media feature is equal to coarse`
7. Leave it entirely implementation defined (really there's subtypes here where we given informative guidance and not, but from a normative point-of-view there's no distinction)

I think (2) and (5) are somewhat obviously non-starters. Between (3) and (6), I'd favour (6) on the basis that it doesn't over-constrain implementations — imagine a WebDriver implementation targeting a browser on a device such as the Nintendo 3DS where the primary pointer subtype is "pen"¹, which feels like a compelling reason to go against any requirement or recommendation in the spec.

I don't think the extreme of (7) is really worthwhile — devices where neither "mouse" nor "touch" are the primary input means are pretty rare!

So I think my preference is (6) — a SHOULD-level requirement, and that doesn't preclude any WPT tests:

This flag should not be used for “SHOULD”; such requirements can be tested with regular tests, like “MUST”.

Footnotes

1. I realise we can debate whether touch or pen is the primary input means on a 3DS, but according to an incredibly unscientific poll the pen is the more common means of input. ↩

[ogarai]

So I think my preference is (6) — a SHOULD-level requirement, and that doesn't preclude any WPT tests:

That sounds reasonable to me!

Another possible set of options I can think of is:

8. Require it to be the same as the primary pointing device in the context of the pointer media feature.

9. Recommend it to be the same as the primary pointing device in the context of the pointer media feature.

These options avoid assumptions about what fine or coarse means.

But I'd say (6) is perfectly fine as well, as it's still a "recommendation" and not a "requirement".

[OrKoN]

I think the option (6) or (9) works for us.

I was re-reading the spec and I think the subtype is currently undefined so this might also be a bug in the spec text?

[gsnedders]

Recommend it to be the same as the primary pointing device in the context of the pointer media feature.

This might be a good idea — we might want to get CSS to export the definition of "primary" here, but that seems reasonable.