Re: [Tracker] [Re: tracker in F16]

[Martyn Russell <martyn lanedo com>]

On 20/09/11 09:49, Martyn Russell wrote:
> Recursive directory change timestamps have also been considered. That
> doesn't fix the issue for people that use USB keys or FAT file systems
> where you can't count on that (which is why we just reindex entire mount
> points recursively to make sure we get everything - and hence why
> Emmanuele Bassi's /home directory would be completely indexed regardless
> of any mtime checks).

I should point out, I did consider that we check the file system type 
and do some interesting things based on that (i.e trust mtime 
information for ext* partitions).

Thus far nothing has happened there though.

> > substantially. Also, tracker appears to BSD lock all files it
> > accesses. That looks quite borked. Which other tool is it synchronizing
> > against here? This looks unsecure to do (because the files are often
> > accessible to others), and since these locks are advisory only there
> > needs to be a strict protocol followed by everybody else accessing these
> > files, which I guarantee you there isn't since these are basically all
> > the user's files. Moreover it appears tracker is mixing BSD and POSIX
> > locks, which is dangerous due to ABBA, in particular when used on NFS
> > directories, which will just end up in total chaos since Linux is so
> > stupid to "upgrade" BSD locks on NFS shares to POSIX locks on the
> > way. In any case you should NEVER EVER use POSIX locking, since it is
> > compltely borked anyway. The locking must go. I also see a massive
> > amount of futex calls in strace, i.e. probably some mutexes thrown in
> > the mix to make the locking problems even more interesting, which makes
> > my fingernails roll up, since they apparently are congested all the
> > time? ]
>
> For now I will assume you mean calls to flock() here and we only do that
> in tracker_file_lock().
>
> While the code we have in place may be improved (and I welcome any
> suggestions you have there), we're actually not using it any more and
> that API could be removed AFAICS:
>
> $ git grep tracker_file_lock .
> src/libtracker-common/tracker-file-utils.c:tracker_file_lock (GFile *file)
> src/libtracker-common/tracker-file-utils.h:gboolean tracker_file_lock
> (GFile *file);

I beg your pardon, I just double checked this and there is also a 
tracker_file_is_locked() API which only the miner-fs uses in one 
instance. That is used when checking each file in our queue handler 
function:

  if (file && queue != QUEUE_DELETED &&
      tracker_file_is_locked (file)) {

I agree, this could potentially be a place to improve. Carlos, 
Aleksander, can you guys comment on why we still have this in place?

--
Regards,
Martyn

Founder and CEO of Lanedo GmbH.