Re: New module proposal: tracker

On 18/08/09 16:07, Lennart Poettering wrote:
On Tue, 18.08.09 13:05, Martyn Russell (martyn lanedo com) wrote:
Hmm. The beef I have with Tracker (and Beagle fwiw) is that they build
something on infrastructure that currently is not good enough
to sustain it: inotify. inotify is simply not suitable for recursively
watching $HOME, but Tracker tries that nonetheless. And that is a big
big failure, it should not do that.

I agree the situation isn't perfect, but it isn't a BIG failure.

Currently Red Hat's Eric Paris is working on this with fanotify:

There are more links from Google of course.

There's something I like to call the "tracker paradox": if you have
a large data set tracker is useless because inotify doesn't scale and
the database is quickly out-of-date -- and if you have a small data
set then you don't need a search engine and hence tracker is useless

Well, that really depends on the user and the data set. Most "normal" users don't have 10 versions of the linux kernel checked out causing these inotify limits to be reached. With ALL my music and external drives I don't have a problem with the limit at all. It seems that only people with the whole of GNOME checked out into $HOME seem to run into these cases.

If I have a small data set (say 10k images), how do I find all images I took in 2006 with tags foo ordered by date? The search engine part is not about just finding "where did I put my images", the querying is much more sophisticated than that.

As I see it the usefulness of Tracker stands or falls with the
scalablity of inotify. As long as inotify does not natively support
recursive watches tracker is not viable.

Well file monitoring really is just one part of Tracker. If you want to keep data from your web applications (say Facebook, etc) in Tracker too, it is completely unrelated.

Right now installing tracker on a non-trivially sized $HOME has the
effect of taking away all inotify watches and thus making inotify
unavailable for other applications.

This is also not true. Tracker never uses ALL resources, it always leaves 512 available for other desktop applications.

-- where they usally are more
appropriately used, such as Nautilus. And all that even without fully
working properly since if the limit of inotify handles is reached the
view tracker has on the file system will necessarily become
out-of-date quickly. Or more drastically spoken: installing Tracker on
a machine with a non-trivially sized $HOME breaks Nautilus and other

I would also argue that some file 20 levels deep in a directory that is never updated is quite unlikely to change and so the chances are quite remote.

We also are using breadth based monitoring so top level folders always get priority here.

And no, increasing the inotify limits is a band-aid, not a fix.


Oh and inotify is not the only area where the Linux file system layer
is not ready to sustain Tracker, it's just the most obvious. Another
thing that come to my mind is that we curently lack recursive mtimes
so that tracker could identify changes on filesystems that happened
while tracker was unable to access them (i.e. due to reboot, logout,
hot unplug).


Tell me about it. I would love to know that some file in folder x changed due to the mtime change at the top most level. But actually, from our experience with the Maemo platform, this is not portable anyway and Windows doesn't even update the current parent's mtime when a file changes in the folder. I would also stress that this likely varies from file system to file system slightly.

Really guys, before pushing forward with getting this into the desktop
make sure that your infrastructure is actually suitable for what you
want to do. And right now it simply is not!

I think you clearly don't understand what Tracker 0.7 offers. It is not ALL about the file system, in fact the file system takes a back seat and is there as a convenience to provide the database with data. Applications also use the data store.

As I see it Tracker is not ready for inclusion in the desktop. It
might not be entirely Tracker's fault though, it's more the kernel's
fault, but that doesn't change the conclusion.

Well, again, that assumes that Tracker is all about file monitoring, which it isn't.

Or in short: just f*ing fix the kernel first.

Are you volunteering? :)


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]