Re: Summary store, impl n. 7

[Philip Van Hoof <me pvanhoof be>]

On Sat, 2007-12-29 at 14:27 -0500, Jeffrey Stedfast wrote:

> The problem with the current summary files is that they don't lend
> themselves well to inserting records out of order or even changing
> record sizes without rewriting the entire summary file :(

It could lend itself to this if all fields where fixed in size and you
mmapped the file as writable.

The problem would of course be locality of data: you can't really let
duplicate strings point to the same address in the mapping, each
non-unique string would be stored non-unique then.

That would badly affect VmRSS (which is what really matters on a small
device imo. Although it was nice to showoff with valgrind massif
reports :-p, valgrind's massif more or less shows malloc and alloca,
which is of course just a small part of the story of memory usage).

> Using a db and records with a bit field representing which pieces of
> info the record has would also be kinda cool if at some later date more
> fields are added to a CamelMessageInfo.


About the idea of using databases...

I thought about the idea of for example using an sqllite. This would be
possible of the API for accessing the summary would strictly be iterator
based (next() and prev(), in stead of accessing it with indexes):

 o. Using a select and then asking a cursor from the database (a cursor
    is like TAIL and HEAD but then smarter and all the hard stuff done
    at the side of the database)

 o. Sorting would always have to be done at the source of the data,
    which means write a smart tree-model that knows how to do this kind
    of sorting in its next() and prev() routines.

 o. Reading all rows would be quite slow compared to what Evolution does
    now

 o. Evolution's VFolder feature would need to be (completely) rewritten
    and the database engine would need a way to merge multiple selects
    together (views on which you request a cursor)

 o. Some things like searching in the summary would be a lot better and
    faster, other things (like most generic operations a user does)
    would probably be slower


gtg now :)



-- 
Philip Van Hoof, freelance software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://pvanhoof.be/blog
http://codeminded.be