Re: New module proposal: tracker

[Alexander Larsson <alexl redhat com>]

On Fri, 2009-11-06 at 15:01 +0000, Martyn Russell wrote:
> > The seeking is hard to avoid, but there may be ways to readdir stuff
> > without having the result be persistant in the buffer cache, although
> > i'm not sure how posix_fadvise(POSIX_FADV_DONTNEED) can be applied when
> > reading a directory...
> 
> Yea. We need to research this a bit more to make sure we are doing 
> everything we can. We did try the posix_fadvise() call and it was fine 
> on the desktop, but for small devices like the n900, it caused a slow 
> down which meant we dropped it.

Hmm, really? In what sense? syscall overhead?

There is one problem with POSIX_FADV_DONTNEED. If you do it on a file
the kernel will drop it from its caches. This is generally what you want
if you just indexed a 100 meg text file that no other app cares about
atm, since it means we won't throw out 100 megs of otherwise useful
caches. However, if you're reading a file that some other app actually
cares about this may be a problem, since you're now ensuring that the
file has to be re-read the next time that app wants to use the file. Not
sure if there is a better way though...

> I should add, we have significantly reduced the number of times we call 
> stat() (from 0.6->0.7). Previously we used stat in trackerd, then again 
> in tracker-indexer, then did something in tracker-extract (which usually 
> involved opening the file). I think we did it several times in some 
> cases too. Just looking at our code again, I think I have found a way to 
> reduce the calls again :)

While this is good news I don't think multiple consecutive calls to stat
on the same file is a huge problem, as all but the first calls will be
availible in the cache already so won't cause i/o.