Re: [Tracker] [Re: tracker in F16]

[Philip Van Hoof <philip codeminded be>]

On Tue, 2011-09-20 at 10:19 +0100, Martyn Russell wrote:
> On 20/09/11 10:11, Philip Van Hoof wrote:
> > On Tue, 2011-09-20 at 11:08 +0200, Aleksander Morgado wrote:

> > > > I beg your pardon, I just double checked this and there is also a
> > > > tracker_file_is_locked() API which only the miner-fs uses in one
> > > > instance. That is used when checking each file in our queue handler
> > > > function:

> > > >     if (file&&  queue != QUEUE_DELETED&&
> > > >         tracker_file_is_locked (file)) {
> > > >
> > > > I agree, this could potentially be a place to improve. Carlos,
> > > > Aleksander, can you guys comment on why we still have this in place?

> > > IIRC, those were just to sync with tracker-writeback, which is (was?)
> > > the only one usign flock()s. Possibly dead code that can be removed.

> > The writeback feature in master has a bug at this moment, so a bit hard
> > to test. But it's true that since its new architecture the flock isn't
> > needed anymore, at least as far as I know.

> Great, care to remove the check and the unused API before this week's 
> 0.12.2 release? :)

I'm first trying to get the writeback bug fixed. But sure, afterward I
can look into removing such old-code flock calls.


Some analysis (me and Aleksander are taking a look at it btw):

In tracker_miner_fs_writeback_notify you can read in a comment that one
needs to look at monitor_item_updated_cb for the final handling of the
task in the writeback_pool.

That's because the writeback program does a rename(), and apparently
it's assumed that this rename is detected by monitor_item_updated_cb
(and that this detection is the end of the writeback procedure)

Normally is tracker_miner_fs_writeback_notify called at the end or reply
of the DBus method to tracker-writeback, but because the rename() on the
FS is detected later than the return of the DBus method, Carlos I think
decided to let the writeback-task really be finally-handled in the
callback that detects said rename() (= monitor_item_updated_cb).

However, although a rename is called (and logged by miner-fs) isn't
monitor_item_updated_cb ever called. And so the writeback_pool is left
with a task.

Now item_queue_handlers_set_up has this code:

/* Already sent max number of tasks to tracker-extract/writeback? */
if (tracker_task_pool_limit_reached (fs->priv->task_pool)  ||
    tracker_task_pool_limit_reached (fs->priv->writeback_pool) ) {
  return;
}

And apparently is that limit, 1. This is causing any second writeback
event to be blocked. And in fs->priv->writeback_pool there's always one
task leaked.

ps. Aleksander thinks that it's maybe related to some event mergings
enabled lately

It's not certain that this problem is also in the 0.12 and 0.10
releases.

Regards,

Philip


-- 


Philip Van Hoof
freelance software developer
Codeminded BVBA - http://codeminded.be