GNOME.org
 

Re: gsk-render review

Hi Alex;

I've been mulling over this for a bit, and started working on what I
think is a feasible solution.

On 14 July 2016 at 22:09, Alexander Larsson <alexander larsson gmail com> wrote:

(Sorry for breaking threading, etc, posting from my phone)

So, I think you misunderstood my opinion. I think the change in API to make
render nodes immutable, etc, is correct. The issue I have is on a higher
level, in the gtk widget tree. Let me try to explain.


Yep, I see what you mean.


Every frame when we draw a toplevel we have each widget in the gtkwidget
tree submit geometry in the form of render nodes. The final result is a
transient immutable entity describing the entire set of rendering operations
for the frame. We can then do complex work on this in the backend to
efficiently render it.


Indeed.


However, submitting geometry shouldn't automatically mean a complete rework
from scratch. We submit a description, which includes references to
textures, vertex arrays, shaders, etc. But we shouldn't have to e.g.
re-upload the textures each time we submit. However, we can't pre-calculate
many of these things ahead of time, even it they are theoretically known by
size-allocate time. For instance, we don't yet have a reference to the gl
context, and we probably want to wait as long as possible to avoid later
size-allocates invalidating the work before render time.



As a simple example, consider a widget that renders just a textured quad.
The texture depends on the size of the widget and some widget state. Lets
consider how this is drawn. By render time the first frame we know the size
and the state, so we can generate and upload the texture data. Then we
create a new render node referencing it and hand it of to the tree. However,
we also keep the texture around, because the next frame we can submit a new
render node referencing the same texture unless something changed.


Yep, I was thinking something along the lines of a texture cache that
is provided by the GSK renderer, but available to GTK via a
specialised API.

Basically, a GskTextureCache would work in terms of surface data,
since this is what GTK already understands, and what GskRenderNode
kind of expects behind the scenes.

If the widget is in invalid state, it asks the texture cache for a
draw surface, draws on it, and then sets the surface as the content of
the render node; additionally, it gets a "cookie" from the texture
cache, so that the widget can get the exact same surface for the
cookie on subsequent frames. The cookie also allows us to keep
references to the surface alive in the texture cache, when we'll move
the rendering off the main thread — thus we can drop a surface from
the cache only when all the frames that have been queued are pushed to
the compositor. The default behaviour would be to drop the surface
reference at the beginning of the render; if the newly submitted
render nodes tree has an additional reference to a texture inside the
cache then the surface will survive; if the contents of the nodes are
different, the surface will be dropped, alongside its GL texture.

If the widget drawing is simple enough that it can be discarded for
every frame, then we can simply call the existing
gsk_render_node_get_drawing_context() instead, and basically get the
exact same behaviour as current GTK+.

The hard part is, as you identified, invalidating the cache — or, at
least, being able to tell a GTK+ widget that its contents have been
invalidated and that it should submit a new surface.


So, what can change? If the widget state changes then we manually mark the
texture invalid and queue a redraw on the widget. This will trigger a
repaint and then we recreate the texture. What if the size changes? Here we
can catch size-allocate and detect a size change (as opposed to a pure move)
and drop the texture. In the move case we just queue a draw, but don't drop
the texture.

Everything in the simple case above can be handled with the things we have
now, but gtk could have APIs to make this simpler. In particular, the
example above is exactly what gtk should do automatically in the case of
falling back to Cairo rendering of a widget, so we need to do this anyway.
I.e I propose adding something between queue-redraw (which just resubmits
geometry) and queue-resize (which requests a layout change). Let's call it
queue-rerender for this mail. If a cairo-using widget is just moved then
queue-redraw is called and the texture from last time we called widget.draw
is reused, but if say the font style or the icon theme changes then
queue-rerender is called and  the old texture is invalidated.


This seems like a sensible approach; the main problem is that we're
now running out of padding slots in the GtkWidgetClass vtable, so this
would either need to be a signal (yuck) or something that cannot be
overridden by subclasses, which kind of conflicts with the point you
make below…


Things also get complicated if a widget wants to do something that is not a
straight rendering of the child nodes. For example it renders the child
widgets into an offscreen and runs a shader on it (another case is efficient
scrolling ala pixel cache). In this case we want to cache things and avoid
rerendering the offscreen. I think you misunderstood this part. I don't mean
that we should keep the render tree between frames. However, if nothing
changes in the widget subtree we can cache the rendered offscreen and reuse
that. The only complexity here is that queue-redraw needs to properly bubble
up the tree so we can catch it and mark the container widget for redraw, and
optionally stop the bubbling (if the child is not visible). This bubbling
currently happens by some ugly low-level gdkwindow callback, and needs to be
brought up to a proper gtkwidget api.


… here.

We may get away with having an internal API for GTK's own widgets,
using a per-class callback inside the class private structure.

My current approach is to provide this "texture cache" to GTK and
ensure that we can simply reuse texture data across frames; this way,
we can also reuse it for implementing the existing pixel caches in use
for TreeView and TextView. We can figure out additional convenience on
top of it inside GTK later on, and see if they can be done without
breaking ABI.

We could also have a way to wrap everything into GtkWidget's own API,
something like:

typedef struct _GtkWidgetDrawCookie GtkWidgetDrawCookie;

cairo_surface_t *
gtk_widget_create_draw_surface (GtkWidget           *widget,
GtkWidgetDrawCookie *cookie);


void
gtk_widget_invalidate_draw_cookie (GtkWidget           *widget,
                                   GtkWidgetDrawCookie *cookie);

If gtk_widget_invalidate_draw_cookie() is not called,
create_draw_surface() will return the same surface for the same
cookie.

After this, we need a way to reliably let a GtkWidget implementation
to call invalidate_draw_cookie() whenever its contents change, as
oppose as its allocation's position. The main problem with this
approach is that it defers to implementations of GtkWidget to decide —
but some operation on GtkWidget itself could affect the cached
surface. For instance, if we change the opacity of a widget we should
just update the opacity of the render node, not the opacity of the
render surface. Similarly, we do have heuristics for determining
whether or not the widget contents should be discarded.

An hybrid approach could be having a flag on GtkWidget that tells it
to always cache its contents, and invalidate them according to
internal heuristics, e.g.:

  gtk_widget_set_cache_content (GtkWidget *widget, gboolean cache);

This would be true for basically every widget; in the case of complex
widgets that need external heuristics, like an explicit
PangoFontDescription change, or the pixel cache, we would allow
developers for fall back to the texture cache API and let them decide
when to invalidate.

Finally, another cache handling API (modeled on CALayer and other
similar scene graph APIs) is to use a declarative model; the contents
of the widget's drawing surface are always cached, and you get to
influence the heuristics on when they should be invalidated via
flags/booleans, e.g.:

  void gtk_widget_set_render_on_allocation_change (widget, bool);
  void gtk_widget_set_render_on_position_change (widget, bool);
  void gtk_widget_set_render_on_size_change (widget, bool);
  void gtk_container_set_render_on_child_change (widget, bool, child);
  void gtk_scrollable_set_render_on_adjustment_change (widget, bool,
range_horiz, range_vert);
  ...

This approach has the advantage of being more extensible without
necessarily requiring subclassing — but at the cost of application
code potentially messing with the API.

Ciao,
 Emmanuele.

-- 
https://www.bassi.io
[@] ebassi [@gmail.com]
[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]