Re: [Tracker] Releasing Tracker 0.6.90

[Carlos Garnacho <carlos imendio com>]

On dom, 2009-02-01 at 20:57 -0500, Jamie McCracken wrote:
> On Fri, 2009-01-30 at 09:46 +0000, Martyn Russell wrote:
> > Jamie McCracken wrote:
> > > > But just when I was happy about one issue being fixed, I noticed that
> > > > performance on removal is awful again:
> > > > Unpacking linux-2.6.28.tar.bz2 in $HOME takes tracker (r2862) around
> > > > 20 minutes to index.
> > > > When I rm -rf linux-2.6.28/, it takes over an hour, with my cpu
> > > > constantly busy (i.e. max speed at 100%) :-/
> > >
> > > yeah thats something that needs investigation prior to release - thanks
> > > for spotting it
> >
> > Carlos is looking into this right now.
> >
> > As it stands, we will be releasing next week unless there are other 
> > issues noticed before then that need addressing.
>
> In addition to deletion of folders :
>
> Other issues: (* mean must fix prior to release)
>
> 1*) directory crawling is too cpu intensive and eats too much memory
> compared to 0.6.6. Old  version did not queue up thousands of files but
> instead queued up directories where mtime indicated it needed updating.
> So memory wise only all directoires (and subdirectories) path were held
> in memory. 

That's too vague, what's the target memory/time constraints? Also, the
crawler doesn't store the files indefinitely, so that's just temporary
memory usage.

> 2*) Moving a file into another directory caused the file to no longer be
> searchable. Also when renaming a directory a search on the new name only
> finds the changed directory name but none of its files or subfolders.
> Tracker should always return a hit if part of the path of a file
> matches! (as per 0.6.6)

Working on that.

> 3*) TST still shows email category twice - Im not sure of cause but i am
> investigating this one

static service_info_t services[17] = {
        { "Emails",        N_("Emails"),       "stock_mail",               NULL, SERVICE_EMAILS,            
NULL, FALSE, 0, 0},
        { "EvolutionEmails",
                           N_("Emails"),       "stock_mail",               NULL, SERVICE_EMAILS,            
NULL, FALSE, 0, 0},
        { "ModestEmails",  N_("Emails"),       "stock_mail",               NULL, SERVICE_EMAILS,            
NULL, FALSE, 0, 0},
        { "ThunderbirdEmails",
                           N_("Emails"),       "stock_mail",               NULL, SERVICE_EMAILS,            
NULL, FALSE, 0, 0},
        ...
};

Notice the second column for all email categories is the same, so they
all get the same translated string in the side pane.

IMHO a nice solution would be to have the side pane transformed to a
tree store, plus a filter to hide children if there's just one of them
(which is the most usual case for mails), and the "root" nodes
(Emails/Files) expanded by default.

Also I really think that a better libtracker API would help improving
quite a lot t-s-t, but please let's leave that after the release :P

> 4*) Disk Io still quite heavy - searching in TST can time out dbus wise
> during indexing. It tries to pause the indexer but takes several minutes
> - i dont suppose there is much we can do here? If not suggest upping
> default throttle levels to minimize the occurance of this

Most of heavy tracker-indexer's operations are asynchronous, so it can
be paused at any stage, the only example where it wouldn't happen is
during heavy (re)move operations (i.e. the linux kernel sources) due to
all the string comparisons in the database and the used collation
function, which is really expensive.

Could you try to get some logs about what's tracker-indexer doing when
pausing times out?

> 5) Applet takes a long time to update its status if started after
> trackerd. 

That happens because it doesn't update until it receives a
index-state-change, perhaps we should also make it a DBus method.

> jamie
>
>
>
>
> _______________________________________________
> tracker-list mailing list
> tracker-list gnome org
> http://mail.gnome.org/mailman/listinfo/tracker-list