Re: New module proposal: tracker

On 18/08/09 17:28, Lennart Poettering wrote:
On Tue, 18.08.09 16:44, Martyn Russell (martyn lanedo com) wrote:

As I see it the usefulness of Tracker stands or falls with the
scalablity of inotify. As long as inotify does not natively support
recursive watches tracker is not viable.

Well file monitoring really is just one part of Tracker. If you want to
keep data from your web applications (say Facebook, etc) in Tracker too,
it is completely unrelated.

And that exists? What providers are there besides the indexer?

Yes, there are some in progress right now.

Right now installing tracker on a non-trivially sized $HOME has the
effect of taking away all inotify watches and thus making inotify
unavailable for other applications.

This is also not true. Tracker never uses ALL resources, it always
leaves 512 available for other desktop applications.

Ah, interesting. It didn't do that the last time I checked. How do you
verify this? by reading the sysctl and then allocating only 512 less
than that? So running tracker means the sysctl is effectively set to
512 from the perspective of other apps?

It has done this since 0.6.9x was released, so for a long time. I think it even did it when Jamie started the code.

This is still an ugly hack, wouldn't you agree?

Sure, but you can't have everything at once and you have to make the best with what you have sometimes.

With a vanilla kernel this leaves 7680 dirs to be watched by
tracker. A single git checkout repo takes away 400 of those here.

Sure, I might not be the typical user, but my music directory already
has>  7000 dirs. My entire $HOME has 82283.


We also are using breadth based monitoring so top level folders always
get priority here.

That's just trying to make the best of a f*ed up situation. It's not
even a work-around, let alone a fix.

So we should just tell users, sorry, the kernel is shit and we can't be arsed to *try* to make things better for you with what's available?

Also I doubt that this logic even ameliorates things. I for one tend
to edit the leaves of my $HOME tree, not the trunk of it. You seem too
assume that this was the other way around for most people. For me at
least the top level directories have few (and seldomly edited) files
in them, the deepest levels most. I mean, do you edit
~/projects/tracker/src/indexer.c more often that ~/indexer.c? (file
names made up, but you get the idea)

Right but source is less important and as I have said earlier in this thread (and others agree) we should probably be ignoring source checkouts.

Tell me about it. I would love to know that some file in folder x
changed due to the mtime change at the top most level. But actually,
from our experience with the Maemo platform, this is not portable anyway
and Windows doesn't even update the current parent's mtime when a file
changes in the folder. I would also stress that this likely varies from
file system to file system slightly.

mjg59 has some ideas about this. Ping him. We were discussing this a
bit at gcds with a couple of folks.

OK, nice thanks.

I think you clearly don't understand what Tracker 0.7 offers. It is not
ALL about the file system, in fact the file system takes a back seat and
is there as a convenience to provide the database with data.
Applications also use the data store.

So you are telling me file system notification does not matter much
for Tracker's usefulness? That's news to me indeed. But of course
leads to the question: what is it then what it offers that desn't need
file system notification? Please explain.

Well, for a start, we don't index . prefixed directories, so for applications that want to store their bookmarks, contacts, etc they can use tracker for that. The Nepomuk ontology describes all of that and how the data relates. Those updates do not come from file system updates, they come from applications sending us data.

I also mentioned internet data being indexed too (i.e. flickr, facebook, etc), we can't use inotify for those can we ;)

Or in short: just f*ing fix the kernel first.

Are you volunteering? :)

I am not hacking on Tracker, am I?

And Tracker is not the kernel either, it is also not the only application that could benefit :)


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]