Re: [Tracker] Search doesn't return anything while content being indexed

[Jimmy Huang <jimmy huang linux intel com>]

On Wed, 2008-08-06 at 11:15 +0300, Ivan Frade wrote:
> Hi Jimmy
>
> On Wed, Aug 6, 2008 at 3:02 AM, Jimmy Huang <jimmy huang linux intel com> wrote:
> > On Wed, 2008-07-30 at 06:02 +0300, Ivan Frade wrote:
> > > Hi Jimmy
> > Ivan,
> >
> > I just checked-out the latest indexer-split branch, compiled and
> > installed it.  It doesn't behave the way you described.  I have some
> > 10,000 images in my home directory to be indexed, I did a re-index and
> > it takes around 10 mins or so to perform the indexing.  While it is
> > being indexed, if I do a search either using the tracker-search-tool or
> > the commandline tracker-search to search for the term "image" or "jpg",
> > it won't return anything.  The tracker stats in the applet gets updated
> > for # of files and folders during indexing, and I can see how many
> > images gets indexed.  But until it finishes indexing all those contents.
> > I can't search any of those.
> >
> > So somehow it is not pausing the indexer and return the search and then
> > continue indexing as you described.  How do I make this work?  Thanks.
>
> It looks like a bug. My first idea is a problem with commits to the
> DB.  Maybe we are not reopening the db connections and they dont read
> the most recent data... We should check the code.
>
>  Thanks for the report. Hopefully next week you can open a bug in bugzilla :)
>
> Ivan

Hi Ivan,

Is this easy to fix in trackerd or it requires substantial code changes?
The reason I am asking is that I am working on building a content
management system on top of tracker, and I need it to do incremental
search/index.  I want to add a timestamp column for each file being
indexed and the ability to incrementally search newly added content
regardless the indexer is crawling or not.  This means assuming I am
indexing a larget set of new files (say using smallest throttle and it
takes 1 hour to index, as the indexer is indexing the new medadata, I
can run a search in a separate thread, it will return me the current
indexed files.  The indexer will continue to index in the background.  I
can then periodically (say every 2 seconds) to perform a search for
anything that's newer than my last search.  Tracker will only return the
set of new files added since the time I did my last search.

I think my two options are:
1) modify tracker to support search while indexing, so pause the
indexer, run search, then continue the indexer (if this support already
exists, and it is easy to fix)

2) write my own client that bypass tracker and read directly from SQLite
database created, this means I open a separate database connection to do
READs while the the tracker is writing.  (is this problematic?)

It would be nice if tracker's architecture already can support this
where I can leverage, or I am thinking that I have to write my own
client to access the DB directly.

Can Jamie or anyone comment on this?

Jimmy