Re: [Tracker] Update for qdbm

[Jamie McCracken <jamie mccrack googlemail com>]

On Fri, 2008-07-25 at 15:10 +0200, Philip Van Hoof wrote:
> On Fri, 2008-07-25 at 09:04 -0400, Jamie McCracken wrote:
> > > With this patch you can update qdbm in src/qdbm for all branches and
> > > TRUNK.
> > >
> > > We are not planning to do this just yet, but I had done it for some
> > > tests anyway, and perhaps in near future it might be useful?
>  
> > that patch removes all my optimizations that I applied to qdbm!
> >
> > IE use of fdatasync instead of fsync and readahead optimisations
>
> Are these optimizations secured against two processes each having their
> own depot instance, but a shared mmap()?

should be

for two processes we need to open the file with file locking

> I can imagine the fsync() stuff being involved with getting this right.
>
> Because I noticed that, in tracker's depot.c, depot->bucket[x] = y is
> only ever done during dpput(), which is a function that is never called
> by a consumer of the depot file that only ever reads.
>
> With two threads that's fine because the same memory for depot and
> depot->bucket is used.
>
> ïThe content of ïdepot->bucket[x] is read by dpget() to see if there's
> something in the bucket. In my tests the content of each and every
> bucket is always 0 for a reading process, but filled up for a writing
> process (if they are different processes).
>
> With two processes there are two allocations for each. Thus I don't see
> how this can work with two processes then :-\
>
> Anyway, we plan to on-flush let the indexer tell trackerd to unload the
> depot file at the next read.
>
> > also adds all the crap that we dont use (my version just included the
> > minimal source we needed)