Re: paint clock

[Soeren Sandmann <sandmann daimi au dk>]

FWIW, I put up the notes I wrote about this subject here:

        http://www.daimi.au.dk/~sandmann/framehandlers.txt

They were written with a different toolkit than GTK+ in mind, so they
make various assumptions that don't apply to GTK+. For example the
widgets are assumed to not do anything like gdk_window_process_exposes().


Soren



Havoc Pennington <hp pobox com> writes:

> Hi,
> 
> I did some doodling around to figure out how GTK could have a "master
> clock" or "paint clock." Some of the problems are a little thorny and
> I need to put this down for a while so I thought I'd throw out some
> work in progress for discussion.
> Unfinished patch attached just to show the direction I was going in,
> it's not intended to be working or even compile.
> 
> Current GTK+ Model
> ===
> 
> GTK+ right now paints in an idle; painting in an idle defers paint
> until after all events have been processed and the request/alloc
> resize idle has run.
> 
> However, painting in an idle fails and looks bad if you are trying to
> do animation or processing a lot of user interaction. If you grep for
> "process_updates" quite a few places in GTK force immediate repaint
> because they are scrolling, animating a GtkImage, drawing a
> rubberband, or interactively resizing a window.
> 
> Painting in an idle is also just plain incompatible with how OpenGL
> and graphics hardware ideally work. What you ideally want to do is
> paint on every vertical refresh, so it's important to not miss a
> refresh while waiting to go idle, and it's important not to paint at
> some random forced time prior to a refresh when you could be doing
> other work.
> 
> Painting in an idle can result in painting too often, not often
> enough, or in an irregular/lumpy way.
> 
> Switching GTK to a paint clock model might have the following benefits:
> * animations could look smooth even without manual forced process_updates
> * multiple animations could sync on the "frame time" used to compute
> tweens on a given frame
> * GTK could more easily be GL-accelerated
> * if the master clock were pluggable, GTK could more easily be
> integrated into another system's paint clock setup, such as Clutter's,
> which would be one way to accomplish the GL acceleration
> * GTK could automatically reduce or increase frame rate according to
> system capabilities or any other consideration
> 
> Alternative to GTK+ Paint Clock
> ===
> 
> Rather than trying to add a paint clock to GTK, a different approach would be:
> 
> * add a way to disable GTK ever using the resize/repaint idles for a
> given widget hierarchy
> * add a way for containers to intercept resize and redraw requests on
> their children (queue_resize, queue_draw). Clutter already has such a
> way - just make queue_draw invoke a thing on the parent instead of
> walking up the widget hierarchy itself.
> * in something like clutter-gtk, the resize/repaint idles would be
> replaced with clutter's
> * the manual process_updates() done by some widgets would also need to
> be gotten rid of someway
> 
> The upside to this alternative: easier to do, less GTK+ breakage.
> The downside: allows fixing GTK+ when embedded in Clutter, but doesn't
> allow fixing GTK+ standalone.
> 
> Clutter-GTK right now
> ===
> 
> Clutter GTK is currently what I'd consider backwards, i.e. it runs
> Clutter paint off the GTK idle. Which downgrades Clutter to GTK's
> behavior.
> http://git.clutter-project.org/clutter-gtk/tree/clutter-gtk/gtk-clutter-embed.c
> 
> In the "Alternative to GTK+ Paint Clock" approach above, GTK+ would
> have the same hook as ClutterActor for replacing queue_redraw which
> would allow flipping how this works so Clutter's master clock would be
> used.
> 
> Clutter Model
> ===
> 
> Clutter used to work in a way not unlike GTK, but now it has a "master clock."
> http://git.clutter-project.org/clutter/tree/clutter/clutter-master-clock.c
> 
> The Clutter master clock can't be lifted wholesale because it isn't
> "model view" but rather is an "omniscient controller" that knows about
> the stuff that needs to be done per tick. As in the attached WIP
> patch, I think the GTK clock should be an interface representing a
> model object, among other reasons because then clutter-gtk could
> implement that interface with the clutter master clock.
> 
> A clutter master clock "iteration" is something like:
> 
> 1. if syncing to vblank, always paint as fast as possible. Otherwise,
> if first paint after sleeping, paint immediately. Otherwise block in
> main loop for remaining frame length since last frame start.
>    (note: without high-res timers added to kernel ca. 2009, poll()
> resolution is 10ms which is half a frame...)
> 
> 2. Force-dispatch all queued X events.
> 
> 3. Update tweens (timelines) with the frame's timestamp, to get all
> actors in the proper state.
> 
> 4. Run some application-added "before repaint" hooks
> 
> 5. Paint and swap buffers
> 
> Step 2 is roughly equivalent to making the X event source have a
> priority higher than G_PRIORITY_DEFAULT.
> 
> Mozilla Model
> ===
> 
> See http://weblogs.mozillazine.org/roc/archives/2010/08/mozrequestanima.html
> 
> The API mozilla uses looks like this:
> 
> var start = window.mozAnimationStartTime;
> function step(event) {
>   var progress = event.timeStamp - start;
>   d.style.left = Math.min(progress/10, 200) + "px";
>   if (progress < 2000) {
>     window.mozRequestAnimationFrame();
>   } else {
>     window.removeEventListener("MozBeforePaint", step, false);
>   }
> }
> window.addEventListener("MozBeforePaint", step, false);
> window.mozRequestAnimationFrame();
> 
> I don't know how they are actually implementing it though, i.e. I
> don't know what they do on each iteration in the guts of their toolkit
> code.
> 
> litl shell model
> ===
> 
> In the litl shell (source not available sadly) we've experimented with
> making things smooth and pretty and fast quite a bit, and landed on a
> model where we dropped g_main_loop_run() in favor of manually doing
> g_main_context_iteration(). Our main loop iterations are like:
> 
> 1. Start a timer.
> 
> 2. Force-dispatch all queued X events up to 5ms of time passed. Stop
> dispatching if we reach 5ms.
> 
> 3. Run one nonblocking main loop iteration (always, even if 5ms passed)
> 
> 4. While the iteration returns TRUE (more to do) and (the timer from
> step 1 is still not up to 5ms OR we don't have a paint queued), do
> more nonblocking iterations.
> 
> 5. If we have any windows using pointer motion hint mask that got a
> motion event since last frame, send async QueryPointer requests. Also
> send some other async X requests. We have to avoid blocking on X round
> trips because with indirect rendering they hose things entirely.
> 
> 6. Update tweens with frame timestamp
> 
> 7. Relayout (request/allocate)
> 
> 8. Block to collect async reply to buffer swap from previous frame
> (using indirect rendering here; collecting the "buffer swapped" event
> in newer direct rendering stuff would have the same effect)
> 
> 9. Send async swap buffers request
> 
> 10. If we need to paint or collect outstanding async requests, goto 1.
> If nothing to do, do a blocking main loop iteration which potentially
> sleeps.
> 
> Another detail of litl shell model is that we always update the frame
> time by the frame length, rather than using wall clock time. This
> looks much better when just painting as fast as possible with a
> nonblocking buffer swap, because you can actually draw three frames in
> say 20ms, but they will display over 60ms. So the tween times should
> be at 20ms intervals not 7ms intervals. We do drop frames but only if
> we get a whole frame behind, and then we still keep the frame times on
> multiples of the frame interval.
> 
> If we were using g_main_loop_run(), what we're doing in effect is
> having the X event source above G_PRIORITY_DEFAULT until 5ms have
> passed, then we dump it down to G_PRIORITY_LOW. The paint source is at
> G_PRIORITY_LOW until 5ms have passed, and then it pops up above
> G_PRIORITY_DEFAULT to run immediately. I'm not sure if we could
> convince g_main_loop_run to do this. The idea of course is to try to
> keep the frame rate up even if we have a flood of dbus messages or
> events or incoming http data or whatever it is.
> 
> We have one other weird detail that would be hard to do with
> g_main_loop_run() which is that when we do a blocking main loop
> iteration, we first do it with a half-second ultra-low-priority
> timeout installed. If that thing fires then nothing happened (not even
> an idle) for half a second. At that point we run a GC and remove the
> timeout, potentially actually sleeping for an indefinite time. This is
> very gjs-specific but perhaps illustrates that applications might want
> to replace the paint clock as well in order to do weird app-specific
> things.
> 
> Adding Paint Clock to GTK+
> ===
> 
> The attached patch shows some initial effort to do this. It introduces
> a simple paint clock object. and initially implements it with an idle
> that works like the current GTK+ idle.
> 
> A problem I'm running into is that the paint clock's semantics just
> don't really support the current GTK+ API:
> * the resize and redraw idle priorities are exposed, and some code
> relies on this to basically do before/after resize/paint hooks.
> * code does manual process_updates to make animations or scrolling or
> resizing look more responsive
> * GtkContainer has the whole "resize mode" API
> 
> Few widgets rely on this stuff in any serious way, but some code does
> rely on it in ways that require at least some fixup.
> 
> In my patch the basic approach is to allow setting a paint clock on a
> GdkWindow and the idea is that the clock applies to all children of
> the window that don't have their own clock set. There's a default
> paint clock used for toplevels with no custom paint clock set.
> 
> I think the patch needs the paint clock to be a bit more involved; it
> needs to be able to control relayout behavior and what happens on a
> manual process_updates, perhaps.
> 
> Some questions to think about:
> * should we do a paint clock at all or just hooks to let containers
> reroute queue_draw/resize coming from their children
> * should the default GTK+ paint clock remain the idle-based approach
> or even without GL should we go to something that's more of a forced
> timeline rather than an idle
> * should manual process_updates even be allowed? drop/no-op this API?
> Or let a window's paint clock decide whether to allow it?
> * does it work to have a separate paint clock on a non-native
> (client-side) window? I should just try it, from the code it isn't
> apparent to me whether painting the parent of a client-side window
> will mess up the pixels of the client-side window. i.e. is it required
> to repaint a client-side window anytime you paint its parent? If not
> should we fix that (clip the client-side window out of parent paint if
> it has a different clock) or should we just say that you can't set a
> clock on a client-side window?
> * should it be possible to override the global default paint clock, or
> only per-window?
> * should the paint clock replace the resize idle also? (I think yes)
> 
> Another note is that I put the clock in GDK, because I think a
> Clutter/GDK backend is interesting and right now the paint idle is in
> GDK also.
> 
> Havoc
> 
> _______________________________________________
> gtk-devel-list mailing list
> gtk-devel-list gnome org
> http://mail.gnome.org/mailman/listinfo/gtk-devel-list