Re: GSoC Report

[John Carr <john carr unrouted co uk>]

Cool!

When I was working on soup tests and i straced conduit to try and
debug a problem and i noticed a side effect of the way we abuse
sys.path: a simple import uuid was causing python to look for various
modules (uuid.pyc, uuid.py, uuid.so, _uuid.so, etc) in every path we
added. I think we should try and reduce our use of sys.path. Instead
of having to add conduit.modules.ThingyModule to path, we should be
able to import directly from conduit.modules.ThingyModule...

John

On Mon, Jun 1, 2009 at 2:47 AM, Alexandre <airmind gmail com> wrote:
> Hi,
>
> Last year's Summer of Code required us to make weekly reports about our
> work, it was good to put things into perspective and to show everybody what
> we were doing. This year there is no such requirement, but I'll try to make
> reports as much as possible.
> This weekend I tried to worked on GSoC, because in the next few weeks I'll
> have very little time.
>
> I used cProfile to look at the Conduit initialization time. In my computer,
> most of time was spent getting device properties from HAL and the rest was
> spent loading all the modules. With 31 files (not counting dependencies) and
> 62 dataproviders getting loaded, things can get slow. And eventually, all we
> need at initialization are factories to find out what new dataproviders to
> create.
> The original HAL factory behaviour when probing for devices was also getting
> problems, because it retrieves all properties from all devices, them it
> checks which are needed. The only module requiring that behaviour, as far I
> could find, is the Synce module. I was able to modify it's behaviour so that
> it is much faster now, but I'm unsure it is working correctly.
>
> The problem of loading all modules at startup I solved by creating a module
> cache. This cache is first created when all modules are loaded and it
> includes the modification time of each file, so we can reload the cache when
> something changes. This cache is kept in the user's config directory. So far
> I could not think a situation where this cache would cause problems, but I'm
> still cautious about it. With the help of the ModuleWrapper architecture in
> place and some hacks, I was able to make Conduit recognize all dataproviders
> without loading any module file and import these modules on demand, when a
> dataprovider instance is required. I also split factories from dataproviders
> in modules that had both of them, so now we can load all factories without
> loading any dataprovider.
> Most of places it is transparent wheter the module has been loaded or it's
> information was in the cache, except in factories, that now have to load
> dataproviders only when required.
> I do not have numbers, but now opening Conduit is much faster and everything
> seems to still be working.
>
> Of course, this is all very experimental. I'll push this work into a branch
> as soon as possible.
>
> Alexandre Rosenfeld
>
> _______________________________________________
> Conduit-list mailing list
> Conduit-list gnome org
> http://mail.gnome.org/mailman/listinfo/conduit-list
>
>