Re: lazy loading override classes

On 9 August 2013 22:53, Johan Dahlin <jdahlin async com br> wrote:

On Friday, August 9, 2013, Daniel Drake wrote:


"from gi.repository import Gtk" naturally triggers the Gtk overrides
file to be imported, and the nature of Python is that this causes a
lot of work to be done. For example, for each class defined there, it
goes ahead and sets up the GObjectMeta, class hierarchies, MRO
calculations, etc.

The GTK overrides file is big and some apps will not even use any of
the hierarchies that are now being prepared. It would be nice if we
could do this lazily.

I am experimenting with a lazy loading approach where the class
definitions are placed in specially decorated functions. Python does
not "descend" into such functions at import time. This causes a 25%
improvement in "import Gtk" timing (which actually brings us down to
pygtk2 import speed).

Great job, nice to see some optimzation love.

pygtk2 is not very efficient, it registers all classes on both c and python
when importing it, including creating a few thousand function wrappers.

It would be interesting to see what gi is doing to try to bring this time

When I first started pygi (or pybank as it was called then) it would import
much instantaneously, but I guess it didnt do many things back then.

Just wanted to point out that I got the Sugar shell running
(experimentally) on top of pygi a few years ago and startup was much
faster and far less memory was consumed. This is to say that whatever
PyGObject is doing today that slows this down isn't really that

And in case someone here hasn't heard this yet, the conclusion I
reached in my Sugar profiling days was that we do far too much
unnecessary stuff. The problem isn't that we do it too slowly, but
that it's far too much unnecessary work and we do it in the worst

Defer that work so it happens at a later stage may be good, but simply
stop doing something that isn't necessary is much better (mainly
because of memory usage, but also because of interactivity).

Today, I would suggest again that compatibility overrides should be
optionally loaded by those users that can afford the penalty, and that
the supported API should be just what PyGObject can offer by relying
solely on the typelibs and GType. This is not only because of
performance, but also because of documentability, consistency and

When I think of the overrides that we ship now as part of PyGObject, I
cannot but wonder if we have really learned the lessons from the years
we had to use and maintain the static bindings.



But that is sort of the point, there shouldnt be a lot of things that _has_
be done, lazily should work for almost everything, including importing

The way to override a class in overrides is now:

def override_Widget():
    class Widget(Gtk.Widget)

     return Widget

There is a new consideration if you want to access the overridden
Widget class from another point in the overrides file - you have to go
up to the DynamicModule instance to find it.

I have ported Gtk and parts of GObject and GLib.

Before I continue with this and open a bug I would be interested in
any quick comments.

Specifically, should I be worried about breaking the API between
overrides and the core, or can I break it without worrying about

Right now it is backwards compatible but for cleanliness and
efficiency I would prefer to remove the old way after porting all

python-hackers-list mailing list
python-hackers-list gnome org

Johan Dahlin
Async Serviços de Informatica Ltda.
Sócio / Diretor
+ 55 16 3376 0125
+ 55 16 9112 6219

python-hackers-list mailing list
python-hackers-list gnome org

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]