Re: [Tracker] tracker-extract not being restarted

[Ralph Böhme <rb netafp com>]

Hi Martyn,

Am 12.05.2014 um 12:26 schrieb Martyn Russell <martyn lanedo com>:

> On 11/05/14 16:13, Ralph Böhme wrote:
> > Hi all,
> >
> > this is somehat a follow up to <https://mail.gnome.org/archives/tracker-list/2014-March/msg00052.html>.
> >
> >   "Shouldn't we restart tracker-extract if it crashes?
> >    I remember one of the reasons moving the data extraction process into
> >    a separate tool was to be more resilient."
> >
> > Looks like on a system tracker-extract hangs and/or crashes and Tracker never recovers. tracker-control -S 
> > keeps reporting tracker-extract as being disabled/not running:
> >
> > # tracker-control -S
> > Store:
> > 09 May 2014, 14:32:47:  ?     Store                 - Idle
> >
> > Miners:
> > 09 May 2014, 14:32:47:    1%  File System           - Crawling recursively
> > directory 'file:///foo/bar'
> > 09 May 2014, 14:32:47:  ?     Extractor             - Not running or is a
> > disabled plugin
> > 09 May 2014, 14:32:47:  ?     Applications          - Idle
> >
> > This is with Tracker 1.0.1.
> >
> > It's easily reproducible by sending tracker-extract a SIGKILL.
>
> I've noticed this too.
>
> I spoke to Carlos about it and we thought about adding a watcher of some kind in tracker-miner-fs.
>
> Though I am not sure if this is the best way forward. We have the same problem with tracker-miner-fs itself 
> and it's less noticed because it will self instantiate on DBus API calls.

But looks like that doesn't help much too and it isn't restarted either.

Just did the following test:
- create a large directory tree with data for Tracker to crawl and index
- stop Tracker: tracker-control -k
- wipe existing db: tracker-control r (ensuring the data I created wasn't indexed behing my back and will be 
crawled next)
- start Tracker: tracker-control -s
- wait until miner is crawling
- kill tracker-miner

root beast:~# tracker-control -k
Found 95 PIDs?
root beast:~# tracker-control -r
Found 95 PIDs?
Setting database locations
Checking database directories exist
Checking database version
  Could not find database version file:'/var/opt/csw/netatalk/tracker/db-version.txt'
  Current databases are either old or no databases are set up yet
  A reindex will be forced
  Creating version file '/var/opt/csw/netatalk/tracker/db-version.txt'
Checking database files exist
Removing all database/storage files
  Removing database:'/var/opt/csw/netatalk/tracker/meta.db'
  Removing journal:'/var/opt/csw/netatalk/tracker/data/tracker-store.journal'
  Removing db-version file:'/var/opt/csw/netatalk/tracker/db-version.txt'
  Removing db-locale file:'/var/opt/csw/netatalk/tracker/db-locale.txt'
root beast:~# tracker-control -s
Starting miners?
  ? File System
  ? Extractor
  ? Applications
root beast:~# tracker-control -S
Store:
12 May 2014, 14:17:12:  ?     Store                 - Idle 

Miners:
12 May 2014, 14:17:12:    0%  File System           - Initializing 
12 May 2014, 14:17:12:  ?     Extractor             - Idle 
12 May 2014, 14:17:12:    0%  Applications          - Initializing 

root beast:~# tracker-control -S
Store:
12 May 2014, 14:17:17:  ?     Store                 - Idle 

Miners:
12 May 2014, 14:17:17:    0%  File System           - Initializing 
12 May 2014, 14:17:17:  ?     Extractor             - Idle 
12 May 2014, 14:17:17:    0%  Applications          - Initializing 

root beast:~# tracker-control -S
Store:
12 May 2014, 14:17:21:  ?     Store                 - Idle 

Miners:
12 May 2014, 14:17:21:    0%  File System           - Initializing 
12 May 2014, 14:17:21:  ?     Extractor             - Idle 
12 May 2014, 14:17:21:    0%  Applications          - Initializing 

root beast:~# tracker-control -S
Store:
12 May 2014, 14:17:24:  ?     Store                 - Idle 

Miners:
12 May 2014, 14:17:24:    0%  File System           - Initializing 
12 May 2014, 14:17:24:  ?     Extractor             - Idle 
12 May 2014, 14:17:24:    0%  Applications          - Initializing 

root beast:~# tracker-control -S
Store:
12 May 2014, 14:17:27:  ?     Store                 - Idle 

Miners:
12 May 2014, 14:17:27:    1%  File System           - Crawling recursively directory 'file:///tank/dbd' 
12 May 2014, 14:17:27:  ?     Extractor             - Idle 
12 May 2014, 14:17:27:  ?     Applications          - Idle 

root beast:~# pkill tracker-miner   
root beast:~# tracker-control -S
Store:
12 May 2014, 14:17:38:  ?     Store                 - Idle 

Miners:
12 May 2014, 14:17:38:  ?     File System           - Not running or is a disabled plugin
12 May 2014, 14:17:38:    7%  Extractor             - Extracting metadata 06m 39s remaining
12 May 2014, 14:17:38:  ?     Applications          - Not running or is a disabled plugin

root beast:~# tracker-control -S
Store:
12 May 2014, 14:17:43:  ?     Store                 - Idle 

Miners:
12 May 2014, 14:17:43:  ?     File System           - Not running or is a disabled plugin
12 May 2014, 14:17:43:   94%  Extractor             - Extracting metadata 01s remaining
12 May 2014, 14:17:43:  ?     Applications          - Not running or is a disabled plugin

root beast:~# tracker-control -S
Store:
12 May 2014, 14:18:15:  ?     Store                 - Idle 

Miners:
12 May 2014, 14:18:15:  ?     File System           - Not running or is a disabled plugin

(tracker-control:2605): Tracker-CRITICAL **: Could not get miner progress for 
'org.freedesktop.Tracker1.Miner.Extract': GDBus.Error:org.freedesktop.DBus.Error.NoReply: Message did not 
receive a reply (timeout by message bus)
Could not get status from miner: org.freedesktop.Tracker1.Miner.Extract12 May 2014, 14:18:27:  ?     
Applications          - Not running or is a disabled plugin

root beast:~# tracker-control -S
Store:
12 May 2014, 14:18:38:  ?     Store                 - Idle 

Miners:
12 May 2014, 14:18:38:  ?     File System           - Not running or is a disabled plugin
12 May 2014, 14:18:38:  ?     Extractor             - Not running or is a disabled plugin
12 May 2014, 14:18:38:  ?     Applications          - Not running or is a disabled plugin

root beast:~# 

Looks like it comes to a grinding halt. :)

How IS tracker-extract actually started these days anyway if not autostarted by DBUS requests?

-Ralph