Re: lazy loading override classes

[Simon Feltman <s feltman gmail com>]

Hi Daniel,
Nice work! Comments below.

On Fri, Aug 9, 2013 at 1:40 PM, Daniel Drake <dsd laptop org> wrote:
> Hi,
> I am experimenting with a lazy loading approach where the class
> definitions are placed in specially decorated functions. Python does
> not "descend" into such functions at import time. This causes a 25%
> improvement in "import Gtk" timing (which actually brings us down to
> pygtk2 import speed).
>
> The way to override a class in overrides is now:
>
> @class_override('Widget')
> def override_Widget():
>     class Widget(Gtk.Widget)
>         [...]
>
>      return Widget
>
> There is a new consideration if you want to access the overridden
> Widget class from another point in the overrides file - you have to go
> up to the DynamicModule instance to find it.

This is fine by me for the majority of overrides. When I first saw
your approach I momentarily thought about if we could use a straight
class decorator with metaclass magic, but I don't think it is possible
and your approach is more obvious. Out of curiosity did you attempt
other techniques or try any alternate approaches?

> I have ported Gtk and parts of GObject and GLib.

I don't think the lazy loading of something like GObject.Object will
help in practice. It might help optimizing isolated import tests, but
everything will need to load the GObject.Object override eventually.
Given that, I would like to avoid the change where we don't think it
will help in real world usage because it adds a tad bit of obfuscation
and destroys the history. We should also definitely note in the commit
message to use "git blame -w" to traverse the history without
whitespace changes. I would break the patches up starting with the new
API/mechanism as a standalone patch and use a new patch for each
override file.

It would also be nice to have regression tests which isolate the lazy
mechanics (not just that overrides work or they don't). So if
something changes later in this bit of code, we can always ensure the
lazy aspect of it continues to work right.

> Specifically, should I be worried about breaking the API between
> overrides and the core, or can I break it without worrying about
> back-compat?
>
> Right now it is backwards compatible but for cleanliness and
> efficiency I would prefer to remove the old way after porting all
> users.

I would need to know more specifics of the API in question
(gi.overrides.override?). But in general we need to keep compatibility
because there are custom overrides which other projects distribute
(GStreamer for example).

Thanks,
-Simon