Re: [Tracker] work items for 0.6.5

[Jamie McCracken <jamiemcc blueyonder co uk>]

On Mon, 2008-02-04 at 16:49 +0800, Halton Huo wrote:
> Jamie, 
>
> As your request, the patch for feature 3) is ready, please review it.

looks mostly right

we do need to index the changed files eventually so can you 

1) add to a separate list all files that have been blacklisted (remove
from the queue as well)

2) check against blacklisted items when changes detected (as we have
removed from queue)

3) trigger a timeout call to index those files after an hour provided
they are not still being changed when they are about to be indexed of
course

thanks

jamie



> Thanks,
> Halton.
> On Fri, 2008-02-01 at 09:52 -0500, Jamie McCracken wrote:
> > > > 3) Constantly changing files - we should ignore these especially
> > > > bittorrent ones. Perhaps keep a small size-limited stack of
> > recently
> > > > indexed files and if one of those files has been changed more than
> > 10
> > > > times in a few minutes we should ignore them until trackerd next
> > > > restarts
> > >
> > > It is like prison rule. 
> > >
> > > Suppose these stack is like:
> > >
> > > file_name      first_change_time     change_accounts
> > > /a             11111                 3
> > > /b             22222                 8
> > > ï/c             33333                 1
> > >
> > > When a file is changed, 
> > >   if ((current_time - ïfirst_change_time) > MAX_DURATION) {
> > >     ïïfirst_change_time = current_time
> > >   } else {
> > >     ïchange_accounts ++;
> > >     if (ïchange_accounts < MAX_CHANGE_TIMES) {
> > >       //reflect this change
> > >     } else {
> > >       // ignore this change
> > >     }
> > >   }
> >
> > thats pretty much it but make sure it applies to files only (and not
> > emails or conversations)
> >
> > when we ignore the file - add it to an ignore list in memory. When
> > trackerd exits, save that list to file. When trackerd is next
> > restarted
> > load the list and index the entries and then reset it.
> >
> > > Stop here if my thinking is totally wrong.
> > >
> > > If my proto is not wrong, the question is where to have this stack?
> > > (1) In memory
> > >    There will have a long list for all changed files, even it is
> > changed
> > > only once.
> > >    
> > > (2) In database
> > >    Add a property for each file in Service table. Could be slower.
> >
> >
> > it would not be slower as such cause we are updating the mtime in that
> > table anyhow whenever it changes. However we dont want to do a db
> > change
> > at this point so (1) would be better for now
> >
> > > Any idea?
> >
> > Use a fixed size LIFO stack of 50 items max in memory 
> >
> > you can use a static array if you like or if you prefer the glib
> > double
> > queue 
> >
> > http://library.gnome.org/devel/glib/unstable/glib-Double-ended-Queues.html)
> >
> >
> > but make sure you pop tail to keep its size limited