Re: [Tracker] [Fwd: Re: ANNOUNCE: tracker 0.7.3 released]

[Martyn Russell <martyn lanedo com>]

On 20/10/09 15:18, Jamie McCracken wrote:
> On Tue, 2009-10-20 at 16:07 +0200, Michael Biebl wrote:
> > 2009/10/19 Carlos Garnacho<carlosg gnome org>:
> > > > A bit more polish (like this) here and there and full text support for
> > > > more file types and it is imho ready for prime time.
> > >
> > > What document formats are you missing specifically?
> >
> > I have a small music collection, ogg encoded, which are properly
> > tagged, but searching for those tags like Album or Title does not
> > return any results in t-s-t.
> > I think I had the same problem with jpegs rsp. tags in their exif header.
> > I also have a lot of documentation as pdf, and for some search terms,
> > which I know exist in the docs, I don't get any result. Not sure if
> > this might be a size related restriction.
> > Excel (xls) support seems to be missing
> > Postscript also.
> > msword/doc seems to work.
> > OpenDocument/odf also.
> > ldif/vcf address book files support seems missing
> >
> > This is from the top of my head.
> >
> > I also had some very bad experience with torrent downloads. tracker
> > seems to be constantly reindexing the file during download and used
> > 100% cpu. I had to exclude/hide the download directory to get a usable
> > system back.
>
> the old tracker maintained a blacklist of frequently changing files to
> avoid this (it would only index 15 minutes after the file in question
> stopped changing) - Im pretty sure this was transferred to new tracker
> but need to check...

That's actually not good enough either.

We have code which should work here, perhaps the config needs tweaking. 
Basically, tracker-miner-fs.cfg has a Monitors section. The two values 
in there of interest are ScanTimeout=0 and CacheTimeout=60. The 
CacheTimeout basically means if we have been getting events for 60 
seconds then we emit an event anyway and continue this every 60 seconds. 
The ScanTimeout is the time that an event MUST wait in the cache before 
being emitted. This is usually 0. The code has a minimum of 1 with 
CACHE_LIFETIME_SECONDS. This means that we will emit events every 1 
second for file changes.

I should add, we changed the notifications recently too. MODIFY events 
should NOT be getting through, we are only interested in:

        mask   = (IN_MOVED_FROM | IN_MOVED_TO | IN_MOVE_SELF |
                  IN_DELETE | IN_DELETE_SELF |
                  IN_CREATE |
                  IN_CLOSE_WRITE |
                  IN_ATTRIB |
                  IN_UNMOUNT);

So it looks like the bit torrent file is being closed every time 
somethign is written too.

Jamie, we no longer have a black list, instead we have a caching system. 
There is a problem with this though - it means we are either quick for 
odd file updates OR we emit a lot of signals (one per second) for files 
with a lot of WRITE/CLOSE events. This is not normal I should add - the 
normal case is we have MODIFY events and one close at the end.

Mikael, can you add your tracker-miner-fs.log here with verbosity set to 
3 please so I can see what is really happening? Thanks.

--
Regards,
Martyn