Re: GTK+ 4.0 and Clutter 2.0: rainbows and unicorns

[Emmanuele Bassi <ebassi gmail com>]

hi Benjamin;

it's going to be a long email... :-)

On 2011-09-06 at 14:26, Benjamin Otte wrote:
> Emmanuele Bassi <ebassi <at> gmail.com> writes:
> > 
> >   a) drop GTK+, move to Clutter and port the complex widges over;
> >   b) re-implement Clutter inside GTK+;
> >   c) use Clutter between GDK and GTK+;
> >
> I would translate that as:
> a) tell GTK developers their code is crap
> b) tell Clutter developers their code is crap
> c) make Clutter and GTK developers work together on the best of both worlds
 
> I have a good idea what solution causes the least tension here... ;)

I know you intended this for comedic effects, but (to be serious for a
second) I don't think ditching either toolkit would be seen as a
judgement on the quality of the code. I know that there are lots of
places in both where the code is crap — but even if we reimplemented
either toolkit on top of the other, a lot of the code would be
salvageable.

the issue is, as usual, is resources. we can pool the (small) resources
we have or just start quasi-from-scratch and take (WARNING: number
pulled out of my arse) five times as much.

> > the Clutter UI description format is JSON and not XML because I honestly
> > don't like XML at all, and GtkBuilder/GtkBuildable was in the wrong
> > position in the stack (GTK instead of GLib) and exposing the wrong API
> > (GMarkup to create custom parsers); on the other hand, ClutterScript
> > made JSON-GLib possible. 
> > 
> I still maintain that the person that writes and maintains the parser and the UI
> for editing gets to decide what format we use. I can handcraft JSON and XML
> files that test GTK. I can even handcraft binary files to break things if I need
> to. And nobody else does care, because they'll all be using the editor anyway.
> (right? :))
> Last but not least, I can also live with JSON existing in Clutter and XML
> existing in GTK. One of them will not work on the GTK level, so 

I totally agree: we need a UI tool, and the UI tool can decided whatever
to use — after all, libglade existed for years outside of GTK and
everything was (almost) fine.

the real issue is the format chosen when a widget (or an actor) want to
intercept the deserialization or the serialization process, and
intervene.

ClutterState's description inside JSON is really ad hoc; same goes for
GtkListStore in XML. you don't want to encode that inside the parsing
code, because that would make third party objects impossible to define.
the crux of the matter is that both the GtkBuildable and ClutterScriptable
interfaces expose the parsing mechanism (and format) inside the API —
GMarkupParser for the former, and JsonNode for the latter.

nowadays we have a serialization/deserialization format in GVariant, but
then again, you'd have to deserialize an on-disk format like JSON or XML
into GVariant first. well, JSON-GLib already can... ;-)

> > Clutter also has some cool concepts like Constraints, which are similar
> > to SizeGroups/Alignments; Actions, which are objects that set up actors
> > to provide high level concepts like click/long-click, gesture
> > recognition, drag and drop; and Effects, which are post-processing
> > filters that use GPU shaders or rendering to offscreen framebuffers.

> So, let's say we think these concepts are awesome and we want them in GTK. In
> particular, actions are something I'd want in GTK preferably right now so that
> when we do the switch in event bubbling, there's not that many event handlers
> left to fix. How do we do that? We don't want to introduce GTK API that is in
> effect identical to the Clutter API. And we don't want to depend on Clutter. Is
> there a solution that can work for this?

no idea. if we don't want to depend on Clutter then we'll have to write
that code somewhere — and the code is tightly coupled with the signals
and with the event handling code. the Actions in Clutter, for instance,
use captured-event on the stage, as well as other API like suspending
the picking when dragging an actor around; or disabling the motion event
throttling when doing gesture recognition; or detecting the actor
underneath the pointer to check for a drop point.

> >   - drop containers and make the scene graph explicitly part of the
> >     ClutterActor API;
> >
> The question of how to present containers in the public API is an interesting
> one. I poked you about this on IRC, so I won't go into the arguments here and
> don't want you to answer, but I'll point it out anyway so people reading this
> can think about it:
> There is 3 consumers of a "container" interface to an Actor/Widget:
> 1) someone implementing a subclass of the Actor/Widget
> 2) application developers using Actors/Widgets and reading API documentation
> 3) development tools like glade
> They all need to be able but not confused about what they are allowed to do and
> what they aren't allowed to do. So group 1 needs the ability to add child actors
> to any actor, but groups 2 and 3 should never do that unless the actor really is
> a container.
> Basically, you want to expose a protected add/remove() function and only make it
> public for real containers. Do we have a way how we support this in GObject?

I see Actors as really 2D layers with nothing really special about them;
any actor can have children (or sub-layers) and you can in theory add a
text label on top of a texture.

what happens with complex widgets? adding a Label on top of a TreeView,
for instance, is something that I honestly would like — I had to use
hacked up sub-classes of GtkVBox that embedded a scrolled window with a
tree view inside, and a label as the second child, and toggled between
the two to show different states. *ghastly*.

to me the distinction between a complex widget and a container is pretty
subtle anyway. you obviously don't want the user to remove sub-widgets
from a composite widget — but that won't change if you have a Container
class or interface. gtk currently assumes that all composite widgets are
really GtkContainers, and it specifically relies on GtkContainer::forall
to return even internal widgets, and you could effectively deconstruct a
GtkFileChooserDefault widget you extracted from a GtkFileChooserWidget —
and you could just do that through the GtkContainer API and a bit of
knowledge of the internals.

> >   - drop Rectangle, Texture, and CairoTexture sub-classes and move
> >     towards a delegate approach for painting the contents of the
> >     actor;
> >   - fully retained painting model using primitives;
> >
> Could you elaborate on the reasons for that a bit more? From looking at WebKit
> (who has a render object/html element split), GTK and Clutter (who don't, but
> apparently want to have) and talking to Tim about Rapicorn[2] (which has very
> few "real" widgets that render themselves and everything else is a container of
> those widgets) I've never managed to find the ideal split here. Because the
> render objects definitely need to be used for layout (their size is kinda
> important) and input (you need to know if you clicked on them after all). And at
> that point they are pretty much full-fledged widgets...

the split as I wrote it in the wip/actor-content branch here:

  https://github.com/ebassi/clutter/tree/wip/actor-content

was:

  • ClutterActor is an empty 2D box that knows where it is going
    to be painted and knows how to react to events;

  • ClutterContent is the code that paints itself given the size
    of the actor.

a Content only has an optional preferred size, otherwise it is expected
to paint itself inside the 3D paint volume.

the branch above was an experiment, mostly; for instance, the preferred
size is not implemented — a Content may only have the size of the
actor's allocation, which defers the sizing to the user of the actor.
it's also true that the only Contents with an implicit size is the Image
content, and an eventual Text content that I never got around
implementing.

also, my latest designs call for primitives instead of raw drawing calls
(the point about "fully retained paint model" above); each Content would
receive a PaintContext generated by Clutter and it'd add primitives, like:

  paint_content.add_texture_rectangle(rect)

or:

  paint_content.add_vertices(verts, colors);

the paint cycle would then collect all the primitives (and eventually
reorganize them to minimize state changes and group the primitives under
the same pipeline state together) and have the actors that issued the
primitives as side-band information, so that when an actor queues a
redraw we know which primitives we need to invalidate, and we can
effectively ask only the interested actors to re-issue them. also, we
could end up doing the work of re-ordering and submitting the geometry
and states inside a separate thread without blocking the main loop.
Cogl's journal is already doing the batching and the pipeline handling,
but we miss something that ties that information to the scenegraph.

so, all in all, there are lost of issues to be solved for that
particular point. :-)

> >   - rework the animation framework, and decide whether or not to
> >     continue using GObject properties.
 
> I'd like some elaboration on the pros and cons here, too. But I can get that
> using beers at the Summit, too. :)

the pros:

  • it eschews completely the property code — though with GProperty we
    can already alleviate a lot of the crap;
  • we can have an "animation context" where actors are effectively
    aware that they are animated, and thus can queue a redraw/relayout
    just once instead of doing that for every property; this would
    cut down the amount of signal emission that we have to perform,
    and that is starting to show in our profiles;
  • it removes a set of API that are not introspectable and generally
    not usable from a language binding perspective.
  • it makes switching between non-animated/animated code easier - for
    instance, this is the API I envision:

      /* start the description of the animation */
      clutter_actor_begin_transition (actor, CLUTTER_EASE_OUT_CUBIC, 250);

      /* set the final state of the actor */
      clutter_actor_set_position (actor, final_x, final_y);
      clutter_actor_set_size (actor, final_width, final_height);

      /* start the animation */
      clutter_actor_end_transition (actor);

    and, additionally:

      /* start a key-frame animation */
      manager = clutter_key_frame_transition_manager_new ();
      clutter_actor_begin_transition_with_manager (actor, manager,
                                                   CLUTTER_LINEAR,
                                                   1000);

      /* first key frame */
      clutter_key_frame_transition_manager_add_key_frame (manager, 0.1);
      /* set the state of the actor at 10% of the transition duration */
      clutter_actor_set_position (actor, node[0].x, node[0].y);
      clutter_actor_set_opacity (actor, 0.5);

      /* second key frame */
      clutter_key_frame_transition_manager_add_key_frame (manager, 0.9);
      /* set the state of the actor at 90% of the transition duration */
      clutter_actor_set_position (actor, node[1].x, node[1].y);
      clutter_actor_set_opacity (actor, 1.0);

      /* start the animation */
      clutter_actor_end_transition (actor);

the cons:

  • is this worth it? we already can do the same with the Animation,
    Animator and State API, though with lots of va_lists, which means
    less type safety and less introspectability;
  • as I said, GProperty already alleviates part of the performance
    issues, but the real kick in the nuts is g_object_notify() being
    generally *awful*.

it's not that I would want to drop the current API — and in theory it
would be possible to implement the property-based API on top of the
non-property-based one — but it's something I really want to
investigate as an option.

+++

I probably forgot something, and I probably opened another ten different
threads...

ciao,
 Emmanuele.

-- 
W: http://www.emmanuelebassi.name
B: http://blogs.gnome.org/ebassi