Re: [Tracker] TRACKER-Critical

[Carlos Garnacho <carlosg gnome org>]

Hi Jose!

On Sun, Apr 16, 2017 at 9:58 AM, Jose Arroyo <jose m arroyo se gmail com> wrote:
> Hello Chris,
>
> It's been a while since I've dived into Tracker so my comments may not be
> terribly useful.
>
> > It seems that whenever I reboot my Ubuntu 16.04.2LTS box I get a huge
> > hourly syslog snippet with what I've put pastebin:
>
>
>  At a first glance, those traces seem to show that Tracker is trying to
> re-index everything after a reboot. I think that this may be triggered by
> some config option but it may be something else.
>
> In any case, the "UNIQUE constraint failed: nie:DataObject.nie:url" error
> means that the SPARQL query that is used to update the file's entry in
> Tracker's DB is trying to insert a second url the the file's entry as if it
> wasn't there. It then fails because there is a constraint to a single url
> per file.
>
> Then you have the logs of this kind: "(tracker-miner-fs:3725):
> Tracker-WARNING **: File
> 'file:///home/chris/Downloads/fetchmail-6.4.0.beta2/trio' has been
> reenqueued more than 2 times. It will not be indexed." The way the Tracker
> DB is organized, an element is not inserted in the DB unless its parent is
> already inserted in the DB. So if for some reason, file://foo/bar is being
> queued for insertion before file://foo, it'll be re-queued for insertion
> after file://foo is inserted.
>
> However, if for some reason, the insertion of file://foo fails, this
> mechanism will start looping. There is a counter to prevent these infinite
> loops which is why you get these logs that say that item X won't be indexed
> because it has already been reenqueued too many times.
>
> This whole behavior is somewhat harmless and, as Sam says, might be fixed by
> reindexing the whole thing from scratch. In any case, what I think is going
> on boils down to 3 things:
> - Something is triggering a re-indexation of already indexed directories on
> reboots (does it also happen if you just stop and start the tracker miner
> fs?). Is it limited only to your "Downloads" directory?
> - Something makes Tracker to try and index found files as if they weren't
> already present in the DB (maybe something wrong with the file last modified
> check/comparison and the held values in Tracker's internal file system
> cache?). This causes the SPARQL insert queries to fail consistently.
> - The reenqueue mechanism triggers the reinsertion of many directories,
> which continue to fail consistently and thus output a large amount of error
> logs.
>
> Btw, for the Tracker maintainers, there has been for some time a missing
> g_object_unref() when a file to be reenqueued is dropped
> (https://git.gnome.org/browse/tracker/tree/src/libtracker-miner/tracker-miner-fs.c#n2134).
> They are ref'd when item_reenqueue() is called, but they are never unref'd
> if the reenqueue'ing isn't actually done. This causes those files to get
> permanently stuck in Tracker's filesystem cache.

Oops, you are right. IIRC you brought this up over IRC but then went
forgotten... I've just pushed
https://git.gnome.org/browse/tracker/commit/?id=900636b2bfe5b91175e521a4acd2296216eb52b0
fixing this.

Thanks for the heads up!
  Carlos