Plans to split Tracker 1.12 (Was: Re: [Tracker] The Big Rip)

[Carlos Garnacho <carlosg gnome org>]

Hi Sam/all,

(CCing distributor-list as suggested by Sam)

On Sun, Oct 30, 2016 at 2:19 PM, Sam Thursfield <ssssam gmail com> wrote:
> Hi!
>
> On 10/28/16, Carlos Garnacho <carlosg gnome org> wrote:
> > It is well known that Tracker has been accumulating lots of code and
> > features over the years, with greatly varying states of maintenance.
> > There's been earlier talks in this ML about splitting the whole thing
> > into more palatable chunks, some refactoring happened towards making
> > this easier, but it was never accomplished.
> >
> > So I'd say now is just as a good time as any to finally do this, I've
> > pushed (so far) the following WIP branches I worked on the last couple
> > of evenings:
> >
> > - wip/split/core : contains Tracker "core", most notably
> > tracker-store, ontologies, libtracker-sparql, libtracker-miner,
> > libtracker-control and the tracker CLI tool.
> > - wip/split/miner-fs : contains everything around FS miner,
> > tracker-miner-fs, tracker-extract and tracker-writeback.
> > wip/split/rss: contains tracker-miner-rss only.
> >
> > The three branches pass distcheck and build stuff into separate
> > tarballs, so (with some rebasing) they're a suitable starting point
> > for standalone repos.
>
> I think we need to be careful about doing this in a way that's useful
> for everyone, and only dong it if there's a good reason. So it would
> be great if some Tracker users & packagers could comment here.

Indeed, I started this thread hoping to get a handful of "please
do/don't"s. There's a lot going on here and it's probably going to
cause some frustration somewhere, I hope for the best in the long
term.

> It might be worth cc'ing the distributor-list to get wider feedback.
>
> What are the use cases of the separated tracker-core and
> tracker-miner-fs repos? I mean, in what situations will someone want
> to use one without the other?

It's true that the most prominently used info from tracker is that
coming from tracker-miner-fs and tracker-extract. Although IMHO that's
far from being the sole purpose, currently Tracker is at its very core
a Nepomuk database.

IMO there's mainly one argument: modularity. Right now Tracker kind of
tries to be an ecosystem by itself, but also attempts not to impose
everything everywhere, so there's this maze of configure script
options which on one hand
adds a combinatorial explosion of code paths, and on the other hand
allows each distributor/packager the choice to pick their own
combination (eg. debian already splits into several packages IIRC, in
others having "tracker" installed is no guarantee that you'll have
/usr/libexec/tracker-miner-rss, while for others is, same for
tracker-preferences/needle).

This way, we avoid --disable-* toggles turning off entire
daemons/modules, provide a clear guideline about how tracker is
structured, and defer the "pick your own pieces" part to either/both
the user or the depends/recommends rules in packages depending on data
from specific miners.

> > Now, the bad news, things are broken, unmaintained or IMHO should be
> > deleted:
> >
> > - tracker-preferences: I'm a bit opinionated on this one,
> > tracker-preferences is a mixed bag of store and miner-fs
> > configuration, and having a GUI to configure a daemon is too 2000's
> > (not just the UI itself...). I think some of its functionality could
> > be moved to the tracker CLI tool, or just be left to DE integration
> > (like gnome search preferences).
>
> It should really be left to desktop environment configuration, indeed...
>
> The existing code is no doubt a good starting point for a
> gnome-control-centre panel. Perhaps we should have some kind of
> tracker-example-apps repo for the existing preferences code?

gnome-control-center's search panel has some minimal tracker
configuration already :) (clicking the gear button) , the only thing
I'm perhaps missing there is configuring the indexing of external
drives, the rest of the stuff in tracker-preferences is bells and
whistles to me.

TBH, I don't see this bunch of code gluing gtkbuilder and gsettings
together that much worthy to preserve...

> It's basically all miner-fs config, I don't see any store
> configuration options there (which seems correct, there's nothing
> useful for end-users to configure about the store).

AFAIR it's just a couple low-level full-text search toggles, I'm
perfectly fine with pointing people to dconf-editor for these tbh...

> > - tracker-needle: This is barely maintained, IMHO we're beyond the
> > times that we needed a standalone/demo GUI, other apps out there make
> > a better work at making Tracker look nice. That said, I know other
> > people saw this useful, so if anyone is willing to maintain it, I'd
> > gladly set up other branch/repo for it.
>
> Again, move to a tracker-example-apps repo?

Yeah I guess.

> > - tracker-miner-apps: AFAIK this is unused since the maemo times, I'd
> > say to send it to the chopper, unless anyone wants to resuscitate it.
> >
> > - tracker-miner-user-guides: ditto.
>
> Maybe the Jolla folk are still using this? If so maybe they could
> adopt them? They're not useful in GNOME to my knowledge.
>
> > - evolution plugin/miner: it's been broken for ages. Again, if anyone
> > cares enough to pick this up and fix it...
> >
> > - firefox and thunderbird plugins: They are basically unmaintained,
> > and AFAIR thunderbird was even blacklisted due to stability issues.
>
> > - nautilus tagging extension: Another one I have a hard time caring...
> > If this feature is as desirable, should be implemented in nautilus
> > itself (and most likely with better looking and more integrated
> > results). I'll probably split it, make a release, and never revisit
> > again. As above, maintainers welcome.
>
> Seems fair, these are definitely not getting enough maintenance to be
> considered part of 'core' tracker.
>
> ...
>
> > The only oddity I see is that the tracker CLI tool (residing in the
> > core) pokes virtually everywhere, it would be nicer to make the
> > "tracker" command a shallow interface to subcommands, that are
> > installed by either tracker core, miner-fs, or whatever.
>
> I spotted that;
>
> > In the future, we would want the ontology out of Tracker core, but
> > probably at the time we can claim tracker-store is a generic SPARQL
> > endpoint :)
> >
> > Comments? Maintainers? I won't rush this much, but would certainly
> > want to tackle this before the branches become too hard to
> > rebase/merge.
>
> I don't really have the desire to become a maintainer, but at the same
> time it seems like you have to do everything right now so I am willing
> to step up my involvement a bit... perhaps I could take on one of the
> repos.

Any help is appreciated :), thanks.

Cheers,
  Carlos