Re: events-delivery branch review - crossing events

[Carlos Garnacho <carlosg gnome org>]

Hi!,

On Mon, Apr 3, 2017 at 10:58 PM, Alexander Larsson <alexl redhat com> wrote:
> So, I took a quick look at the event-delivery branch. One fundamental
> thing that it is currently missing is the handling of crossing events
> due to size allocation changes. In the simple case this is just a
> label changing context, making the widget wider which changes the
> widget under the pointer.
>
> However, there is also some real complexity in this. For instance,
> it's possible to get loops this way, such as if the :hover state
> causes the new widget position to be outside the pointer.
>
> If you look at how CSS handles this, the specification says:
>
>   User agents are not required to reflow a currently displayed
>   document due to pseudo-class transitions. For instance, a style
>   sheet may specify that the 'font-size' of an :active link should be
>   larger than that of an inactive link, but since this may cause
>   letters to change position when the reader selects the link, a UA
>   may ignore the corresponding style rule.
>
> However, testing firefox and chrome[1] it seems in practice what
> happens is that :hover causes a reflow, but the new reflow does not in
> turn cause re-calculating the :hover state (until the next mouse event
> at least). This seems like like a pretty nice behaviour in a weird
> situation.
>
> In the  current pre-event-delivery branch  what happens in  the layout
> cycle is this:
>
> 1) emit all events in the queue then freeze the event queue
> 2) emit update which triggers all animations etc
> 3) Do size machinery, possibly loop up to 4 times
> 4) Paint
> 5) Unfreeze events, possibly queueing a new frame
>
> In the above, a mouse enter event in 1 would cause a css property
> change, which would cause the cause the relayout in 3 to produce a new
> GdkWindow geometry, which in turn will emit new enter/leave events
> when the queue is unfrozen at 5 and which cause a new cycle the next
> frame. The frame clock will limit the cpu use from 100%, but its still
> not ideal to constantly switch between two states. Also, even if we
> don't get a loop the correct rendering is always delayed one frame.
>
> The question is how to handle this in the new model. The naive version
> would cause the hover css property to change immediately from size
> allocate, which will cause a layout loop that runs 4 times, and then
> paint. Another alternative is to keep a crossing event for this so
> that we can store it on the event queue, and this way we can reproduce
> the current behaviour. However, that strikes me as non-ideal too.
>
> An alternative would be to treat crossing events as level-triggered
> instead of edge-triggered, at most once a frame. Every frame, after
> the first iteration in the layout machinery we pick the current
> position for all pointers, and emit css state changes (as well as
> generic widget event callbacks). If any of these queue a resize we'll
> handle these in the next iteration, but we never generate further
> crossing events this frame, nor do we automatically schedule a new
> frame just due to this.

This was roughly my idea too, and I think it fits in nicely with frame
clock based drawing, we certainly want to limit the cpu time necessary
to handle the aftereffects of input events to a minimum.

> This has some complications in semantics though. If you move the
> pointer between multiple widgets in one frame we will miss some
> crossing events that we would otherwise have seen. I don't think this
> is a problem in itself, because these would unlikely have resulted in
> something that would affect the final frame result (it would
> essentially be like the user moved the mousepointer so fast that it
> jumped over the inbetween widget completely). However, it is not
> entierly clear how to report the motion events that land in-between
> the two widgets that had the enter notify reported to them. Getting
> motion events without enter events is quite a change compared to
> the current semantics. Can we just drop these events?

IMHO, if we go down this path we can and should drop them, it's not
coherent to report motion to widgets that didn't previously receive an
enter event. As you say this is not too different from quick pointer
motions, or touchscreen-driven pointer on x11. It'd just appear as if
the pointer warped from one place to the other. And we'd just be
rate-limiting it to 60fps.

For motions alone, I think this shouldn't matter much in practical
terms, plain hovering with no interaction should just cause visual
feedback most often[1], just like crossing events [2], and even more
so if the pointer is/gets grabbed somewhere else. This is more unclear
if the user manages to click in between, as we should arbitrate which
widget actually received the button press and implicitly grab on it.

What I'd suggest is:
1) All arriving input events are queued in toplevel coordinates.
2) When the frame clock says it's time to handle events:
  - Coalesce the queue to a minimal series of events, might result in
> 1 motions if there's button presses in between
  - Emit those ensuring the correct/orderly crossing event emission
given the coalesced motions (which should still be one set of crossing
events given the appearance of grabs)
3) For widgets/controllers that want the full motion history, keep the
original events around during the event delivery phase so we can
reconstruct the event history for these widgets within motion
handlers.

Things complicate a bit further I guess if we account for
press+release within the same frame time (Which I don't think is
possible, except programmatically). In this case the resulting
implicit grab kind of helps despite relayouts (although whether the
current target widget is correct for the button release is debatable),
style wise I think it's fair to say "toggling an state on and off
within the same frame may not result in visible effects", and event
management wise both events  would be delivered, so all "permanent"
effects should still apply.

As per these ideas, event compression happens by default throughout
the widget hierarchy with no off switch whatsoever, so should be
compensated by accessors to let motion handlers in gestures/widgets
rebuild the set of x+y+time out of the uncompressed events.

Cheers,
  Carlos

[1] FWIW, GtkGesture just listens to press/motion*/release sequences.
I think we'll need some "motion capturer" event controller, but it
would be a GtkEventController rather than a GtkGesture, so it'll have
no means to claim pointer input for itself, grabs aside.
[2] I can think other less common usecases where missing all
crossing/motion might bring unexpected effects, like
capturing/confining the pointer whenever it enters some area, however
I'd expect/trust these to be big enough that they can't be crossed in
sub-frame time.