Re: [Tracker] tracker-extract not being restarted

[Philip Van Hoof <philip codeminded be>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martyn Russell schreef op 12/05/2014 16:18:
> On 12/05/14 15:06, Ralph Böhme wrote:
> > Hi Martyn

[cut]

> We used to use DBus and that would instantiate processes for us,
> but now we have this "passive" approach. It has advantages and
> disadvantages.

It has mostly advantages in my opinion. Also wrt watchdogging.
Although we're no longer implementing it apparently ;) (which is a
problem)

> The advantage is, tracker-miner-fs is simpler and doesn't have to
> deal with the mess of broken files crashing tracker-extract and
> timeouts, etc. Also it means the file basic information is added to
> Tracker quicker than it was previously because we have no delay on 
> tracker-extract. Another advantage half realised (bits can be
> improved still), is that we can group process files of the same
> time and prioritise files of particular types. We have mime type
> priorities currently I think. Disadvantages include having to wait
> for the second pass indexing to complete :)
>
> tracker-extract has _ALWAYS_ crashed because of the nature of the
> files and libraries we're using being the weakest point. With this
> in mind, it's quite bad that we don't have some sort of watchdog or
> way to keep it up and running.

I think the watchdog should be configurable per system configuration.

For example with systemd we can let systemd automatically manage (and
that includes restarting) the process in case of a crash.

On systems that don't use systemd they'll probably have another init
subsystem that can do managent of this.

I think that both for systemd-based and for those systems it would be
silly to insist on using our own implementation of a separate watchdog
or forever-loop script to ensure tracker-extract is kept up:

        Often the sysadmin of such a system want the system-wide watchdog or
init infastructure to be aware of these restarts of the process.

Although I understand it smells like %windir%\system32\services.msc
and although the panic caused by this is becoming pandemic on
slashdot, I still think this is the right architecture nonetheless.

Because even our original implementation where tracker-extract was
activated because of a 'active' D-Bus method call was not perfect in
that there was no awareness or control by the system's infrastructure
to avoid that a constantly crashing tracker-extract is returning all
the times (what you later refer to as circular problems).

        Except that miner-fs would in the original implementation after
several attempts skip the file causing the crash (which was, fair
enough, quite a good attempt at somewhat solving the situation).

Nonetheless should we outsource this logic to the init system or the
"CoreOS" infrastructure as Lennart calls it (the kid has a name now).

On a typical Linux in the future this will hopefully be systemd, and
in my opinion we only need to provide the hooks and configurability to
system integrators who plan to use a different system for this purpose
(with default implementations for systemd, and maybe also for upstart
and openrc or whatever popular-alternative init system the knigths who
say NI(H) will use).

An example reason why I think the Core-OS subsystem should do this is
because I also believe that tracker-extract should run in its own
lightweight container, precisely because it can crash (because a crash
on input like a file is often an indication of a serious security
problem -- a reason why it should run in a container in my opinion).

Infrastructure like systemd could in case of a tracker-extract crash
revert the copy-on-write filesystem behind the container and restart
the entire OS, or it could opt to just restart the tracker-extract
process. The strategy to use isn't something the Tracker project
should choose, but the init system could (based on the sysadmin's
choices).


> We could easily do this in tracker-miner-fs. Part of

But I think we shouldn't in case systemd is already doing it.

> me thinks we should actually have a tracker-control --daemon which 
> watches all process and keeps some sort of journal / log going and
> also maintains processes like tracker-extract keep running. I
> should add, this is something that nearly EVERY embedded solution
> does itself with some script which keeps miner-fs running (not even
> tracker-extract), because there is no other way to guarantee
> Tracker is there to catch all events.

That script is what systemd should do.

> The only downsides I see are repeated attempts causing a circular 
> problem with tracker-extract. But that's easily remedied, we had to
> do something like this in tracker-miner-fs before already.

Circular problems like this is what systemd would solve.


> Happy to help :)

Cheers :)

Philip



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJTcek/AAoJEEP2NSGEz4aDRHwIAJQTXt2eM9pZTZqwT/Z1Po/v
19/mFxwTA+ek57G0Tib85vx6TMWLjWQa295pC7dOpvI3cWDlEJtD56uHBg5IFOED
lV1+qL3Kt0pjklqaczWDoIXmlDA2jJ2f4dqYNalbFU7YEd/6sZyOo42C/bI8eBCX
+scDn5VaCcW51Y1nvAzEs9yaulN18qZQCuPLRqEZOCwXB0Dy08JFQnOjC7+bl/gc
F6sDIOdcmS9Qx2gyJv3x6r7X2AbJb0kJvui88/Q0IDvElxqy5w60yn8m7YTIHSqS
Oauw1Ytbdy0ygxqtL1ljtfJnT1BxPiVj9mr+zDrSIkAAa9DgsQ+9wNwNnGbZ4Ww=
=qOb/
-----END PGP SIGNATURE-----