[weekly report] week 1: conduit

[Alexandre <airmind gmail com>]

Hi all,

[I sent this to the Conduit mailing list, so I'm just forwarding it here]

I used cProfile to look at the Conduit initialization time. In my computer, most of time was spent getting device properties from HAL and the rest was spent loading all the modules. With 31 files (not counting dependencies) and 62 dataproviders getting loaded, things can get slow. And eventually, all we need at initialization are factories to find out what new dataproviders to create.
The original HAL factory behaviour when probing for devices was also getting problems, because it retrieves all properties from all devices, them it checks which are needed. The only module requiring that behaviour, as far I could find, is the Synce module. I was able to modify it's behaviour so that it is much faster now, but I'm unsure it is working correctly.

The problem of loading all modules at startup I solved by creating a module cache. This cache is first created when all modules are loaded and it includes the modification time of each file, so we can reload the cache when something changes. This cache is kept in the user's config directory. So far I could not think a situation where this cache would cause problems, but I'm still cautious about it. With the help of the ModuleWrapper architecture in place and some hacks, I was able to make Conduit recognize all dataproviders without loading any module file and import these modules on demand, when a dataprovider instance is required. I also split factories from dataproviders in modules that had both of them, so now we can load all factories without loading any dataprovider.
Most of places it is transparent wheter the module has been loaded or it's information was in the cache, except in factories, that now have to load dataproviders only when required.
I do not have numbers, but now opening Conduit is much faster and everything seems to still be working.

Of course, this is all very experimental. I'll push this work into a branch as soon as possible.

Alexandre Rosenfeld