Re: New module proposal: tracker

[Lennart Poettering <mztabzr 0pointer de>]

On Tue, 18.08.09 13:05, Martyn Russell (martyn lanedo com) wrote:

> Hi all,
>
> So we recently polled the tracker mailing list to make sure the core  
> developers and others interested had an opinion on GNOME module  
> inclusion for Tracker. You can see the thread here:
>
> http://mail.gnome.org/archives/tracker-list/2009-August/msg00007.html
>
> The response was positive. So I would like to propose Tracker as a new  
> GNOME module.

Hmm. The beef I have with Tracker (and Beagle fwiw) is that they build
something on infrastructure that currently is not good enough
to sustain it: inotify. inotify is simply not suitable for recursively
watching $HOME, but Tracker tries that nonetheless. And that is a big
big failure, it should not do that.

There's something I like to call the "tracker paradox": if you have
a large data set tracker is useless because inotify doesn't scale and
the database is quickly out-of-date -- and if you have a small data
set then you don't need a search engine and hence tracker is useless
too.

As I see it the usefulness of Tracker stands or falls with the
scalablity of inotify. As long as inotify does not natively support
recursive watches tracker is not viable.

Right now installing tracker on a non-trivially sized $HOME has the
effect of taking away all inotify watches and thus making inotify
unavailable for other applications -- where they usally are more
appropriately used, such as Nautilus. And all that even without fully
working properly since if the limit of inotify handles is reached the
view tracker has on the file system will necessarily become
out-of-date quickly. Or more drastically spoken: installing Tracker on
a machine with a non-trivially sized $HOME breaks Nautilus and other
software.

And no, increasing the inotify limits is a band-aid, not a fix.

Oh and inotify is not the only area where the Linux file system layer
is not ready to sustain Tracker, it's just the most obvious. Another
thing that come to my mind is that we curently lack recursive mtimes
so that tracker could identify changes on filesystems that happened
while tracker was unable to access them (i.e. due to reboot, logout,
hot unplug).

Really guys, before pushing forward with getting this into the desktop
make sure that your infrastructure is actually suitable for what you
want to do. And right now it simply is not!

All these things are fixable. Apple has shown that one can get all
these things fixed. It's just a matter of someone sitting down and
actually fixing the kernel. But as long as that hasn't happened you
are roofing your house before you built its foundation.

As I see it Tracker is not ready for inclusion in the desktop. It
might not be entirely Tracker's fault though, it's more the kernel's
fault, but that doesn't change the conclusion.

Or in short: just f*ing fix the kernel first.

Lennart

-- 
Lennart Poettering                        Red Hat, Inc.
lennart [at] poettering [dot] net
http://0pointer.net/lennart/           GnuPG 0x1A015CC4