Re: lazy loading override classes

[Tomeu Vizoso <tomeu tomeuvizoso net>]

On 9 August 2013 22:53, Johan Dahlin <jdahlin async com br> wrote:
> On Friday, August 9, 2013, Daniel Drake wrote:
> > Hi,
> >
> > "from gi.repository import Gtk" naturally triggers the Gtk overrides
> > file to be imported, and the nature of Python is that this causes a
> > lot of work to be done. For example, for each class defined there, it
> > goes ahead and sets up the GObjectMeta, class hierarchies, MRO
> > calculations, etc.
> >
> > The GTK overrides file is big and some apps will not even use any of
> > the hierarchies that are now being prepared. It would be nice if we
> > could do this lazily.
> >
> > I am experimenting with a lazy loading approach where the class
> > definitions are placed in specially decorated functions. Python does
> > not "descend" into such functions at import time. This causes a 25%
> > improvement in "import Gtk" timing (which actually brings us down to
> > pygtk2 import speed).
>
>
> Great job, nice to see some optimzation love.
>
> pygtk2 is not very efficient, it registers all classes on both c and python
> side
> when importing it, including creating a few thousand function wrappers.
>
> It would be interesting to see what gi is doing to try to bring this time
> down.
>
> When I first started pygi (or pybank as it was called then) it would import
> pretty
> much instantaneously, but I guess it didnt do many things back then.

Just wanted to point out that I got the Sugar shell running
(experimentally) on top of pygi a few years ago and startup was much
faster and far less memory was consumed. This is to say that whatever
PyGObject is doing today that slows this down isn't really that
necessary.

And in case someone here hasn't heard this yet, the conclusion I
reached in my Sugar profiling days was that we do far too much
unnecessary stuff. The problem isn't that we do it too slowly, but
that it's far too much unnecessary work and we do it in the worst
moment.

Defer that work so it happens at a later stage may be good, but simply
stop doing something that isn't necessary is much better (mainly
because of memory usage, but also because of interactivity).

Today, I would suggest again that compatibility overrides should be
optionally loaded by those users that can afford the penalty, and that
the supported API should be just what PyGObject can offer by relying
solely on the typelibs and GType. This is not only because of
performance, but also because of documentability, consistency and
maintainability.

When I think of the overrides that we ship now as part of PyGObject, I
cannot but wonder if we have really learned the lessons from the years
we had to use and maintain the static bindings.

Regards,

Tomeu

> But that is sort of the point, there shouldnt be a lot of things that _has_
> to
> be done, lazily should work for almost everything, including importing
> dependencies.
>
> > The way to override a class in overrides is now:
> >
> > @class_override('Widget')
> > def override_Widget():
> >     class Widget(Gtk.Widget)
> >         [...]
> >
> >      return Widget
> >
> > There is a new consideration if you want to access the overridden
> > Widget class from another point in the overrides file - you have to go
> > up to the DynamicModule instance to find it.
> >
> > http://dev.laptop.org/~dsd/20130809/class_override.patch
> >
> > I have ported Gtk and parts of GObject and GLib.
> >
> > Before I continue with this and open a bug I would be interested in
> > any quick comments.
> >
> > Specifically, should I be worried about breaking the API between
> > overrides and the core, or can I break it without worrying about
> > back-compat?
> >
> > Right now it is backwards compatible but for cleanliness and
> > efficiency I would prefer to remove the old way after porting all
> > users.
> >
> > Thanks
> > Daniel
> > _______________________________________________
> > python-hackers-list mailing list
> > python-hackers-list gnome org
> > https://mail.gnome.org/mailman/listinfo/python-hackers-list
>
>
>
> --
> Johan Dahlin
> Async Serviços de Informatica Ltda.
> Sócio / Diretor
> + 55 16 3376 0125
> + 55 16 9112 6219
>
> _______________________________________________
> python-hackers-list mailing list
> python-hackers-list gnome org
> https://mail.gnome.org/mailman/listinfo/python-hackers-list