Re: [pygtk] Overriding GObject methods in Python

[Hrvoje Nikšić <hrvoje niksic avl com>]

On Thu, 2006-11-23 at 17:20 +0100, Johan Dahlin wrote:
> You seem to be very worried about performance. 

Only for low-level code, where C would be used for efficiency.  Ideally,
calling C methods from C code should be as fast as invoking a function
via a function pointer (possibly achieved by caching the method lookup).
Calling Python from C or C from Python can involve hashtable lookups,
performance is obviously not a priority there.

> I advise you to try it out and see if it meets your requirements.
> 
> If it doesn't, rewrite the whole thing in C.

Your advice is sound and it works well for most projects.  For some
larger projects a bit more planning is needed because by the time the
project reaches a prototype stage, it might be too late to rewrite it in
C.

In the case of the project I'm working on, the prior code base was
written in Scheme, using an interpreter I will call "Morass" to protect
the innocent.  An object system was designed on top of Morass's CLOS
implementation, and Morass's GTk bindings were bound to these custom
objects to form the GUI layer.  Authors paid little attention to
performance during design and implementation, planning instead to make
it work first and then rewrite the components that proved too slow in C,
on a case-by-case basis.  By the time a working prototype was reached, a
large chunk of code was already written.  Unfortunately, the prototype
turned out to be very slow: it loaded for a long time, and the GUI that
used it proved too sluggish to use.  After many attempts to profile and
optimize it, it was still too slow.  And this was only in a small
program that was written to demonstrate the system, which was supposed
to also be usable in a wide range of applications.

At this point, it was definitely too late for a full rewrite in C.  The
system was written in a purely interpreted dynamic language augmented
with a highly flexible dynamic OO system customized with MOP.  It
created a multiple of objects which were in reality heavyweight due to
the dynamic nature of the language.  It was far from obvious which parts
to "rewrite in C" to get a measure of speed.  Rewrite Morass's reader to
get better loading performance?  Implement a compiler for Morass to get
better run-time performance?  Rewrite implementation of some CLOS
objects in C?  Given the nature of the underlying technology and the use
of MOP, any one of those projects would be a major undertaking, and it
is questionnable whether any one alone would significantly boost
performance.

The moral of the story is not premature optimization, nor is it my
intention to imply that Python is as slow as Morass (it's in fact
significantly faster, although not by an order of magnitude).  The point
is that performance has to be at least considered when choosing the
underlying technology and design, because the underlying technology is
hard to change later without a major rewrite.