Re: Plans to split Tracker 1.12 (Was: Re: [Tracker] The Big Rip)

[Bastien Nocera <bnocera redhat com>]

----- Original Message -----
> Hi Sam/all,
>
> (CCing distributor-list as suggested by Sam)
>
> On Sun, Oct 30, 2016 at 2:19 PM, Sam Thursfield <ssssam gmail com> wrote:
> > Hi!
> >
> > On 10/28/16, Carlos Garnacho <carlosg gnome org> wrote:
> > > It is well known that Tracker has been accumulating lots of code and
> > > features over the years, with greatly varying states of maintenance.
> > > There's been earlier talks in this ML about splitting the whole thing
> > > into more palatable chunks, some refactoring happened towards making
> > > this easier, but it was never accomplished.
> > >
> > > So I'd say now is just as a good time as any to finally do this, I've
> > > pushed (so far) the following WIP branches I worked on the last couple
> > > of evenings:
> > >
> > > - wip/split/core : contains Tracker "core", most notably
> > > tracker-store, ontologies, libtracker-sparql, libtracker-miner,
> > > libtracker-control and the tracker CLI tool.
> > > - wip/split/miner-fs : contains everything around FS miner,
> > > tracker-miner-fs, tracker-extract and tracker-writeback.
> > > wip/split/rss: contains tracker-miner-rss only.
> > >
> > > The three branches pass distcheck and build stuff into separate
> > > tarballs, so (with some rebasing) they're a suitable starting point
> > > for standalone repos.
> >
> > I think we need to be careful about doing this in a way that's useful
> > for everyone, and only dong it if there's a good reason. So it would
> > be great if some Tracker users & packagers could comment here.
>
> Indeed, I started this thread hoping to get a handful of "please
> do/don't"s. There's a lot going on here and it's probably going to
> cause some frustration somewhere, I hope for the best in the long
> term.
>
> > It might be worth cc'ing the distributor-list to get wider feedback.
> >
> > What are the use cases of the separated tracker-core and
> > tracker-miner-fs repos? I mean, in what situations will someone want
> > to use one without the other?
>
> It's true that the most prominently used info from tracker is that
> coming from tracker-miner-fs and tracker-extract. Although IMHO that's
> far from being the sole purpose, currently Tracker is at its very core
> a Nepomuk database.
>
> IMO there's mainly one argument: modularity. Right now Tracker kind of
> tries to be an ecosystem by itself, but also attempts not to impose
> everything everywhere, so there's this maze of configure script
> options which on one hand
> adds a combinatorial explosion of code paths, and on the other hand
> allows each distributor/packager the choice to pick their own
> combination (eg. debian already splits into several packages IIRC, in
> others having "tracker" installed is no guarantee that you'll have
> /usr/libexec/tracker-miner-rss, while for others is, same for
> tracker-preferences/needle).
>
> This way, we avoid --disable-* toggles turning off entire
> daemons/modules, provide a clear guideline about how tracker is
> structured, and defer the "pick your own pieces" part to either/both
> the user or the depends/recommends rules in packages depending on data
> from specific miners.
>
> > > Now, the bad news, things are broken, unmaintained or IMHO should be
> > > deleted:
> > >
> > > - tracker-preferences: I'm a bit opinionated on this one,
> > > tracker-preferences is a mixed bag of store and miner-fs
> > > configuration, and having a GUI to configure a daemon is too 2000's
> > > (not just the UI itself...). I think some of its functionality could
> > > be moved to the tracker CLI tool, or just be left to DE integration
> > > (like gnome search preferences).
> >
> > It should really be left to desktop environment configuration, indeed...
> >
> > The existing code is no doubt a good starting point for a
> > gnome-control-centre panel. Perhaps we should have some kind of
> > tracker-example-apps repo for the existing preferences code?
>
> gnome-control-center's search panel has some minimal tracker
> configuration already :) (clicking the gear button) , the only thing
> I'm perhaps missing there is configuring the indexing of external
> drives, the rest of the stuff in tracker-preferences is bells and
> whistles to me.
>
> TBH, I don't see this bunch of code gluing gtkbuilder and gsettings
> together that much worthy to preserve...
>
> > It's basically all miner-fs config, I don't see any store
> > configuration options there (which seems correct, there's nothing
> > useful for end-users to configure about the store).
>
> AFAIR it's just a couple low-level full-text search toggles, I'm
> perfectly fine with pointing people to dconf-editor for these tbh...
>
> > > - tracker-needle: This is barely maintained, IMHO we're beyond the
> > > times that we needed a standalone/demo GUI, other apps out there make
> > > a better work at making Tracker look nice. That said, I know other
> > > people saw this useful, so if anyone is willing to maintain it, I'd
> > > gladly set up other branch/repo for it.
> >
> > Again, move to a tracker-example-apps repo?
>
> Yeah I guess.
>
> > > - tracker-miner-apps: AFAIK this is unused since the maemo times, I'd
> > > say to send it to the chopper, unless anyone wants to resuscitate it.
> > >
> > > - tracker-miner-user-guides: ditto.
> >
> > Maybe the Jolla folk are still using this? If so maybe they could
> > adopt them? They're not useful in GNOME to my knowledge.
> >
> > > - evolution plugin/miner: it's been broken for ages. Again, if anyone
> > > cares enough to pick this up and fix it...
> > >
> > > - firefox and thunderbird plugins: They are basically unmaintained,
> > > and AFAIR thunderbird was even blacklisted due to stability issues.
> >
> > > - nautilus tagging extension: Another one I have a hard time caring...
> > > If this feature is as desirable, should be implemented in nautilus
> > > itself (and most likely with better looking and more integrated
> > > results). I'll probably split it, make a release, and never revisit
> > > again. As above, maintainers welcome.
> >
> > Seems fair, these are definitely not getting enough maintenance to be
> > considered part of 'core' tracker.
> >
> > ...
> >
> > > The only oddity I see is that the tracker CLI tool (residing in the
> > > core) pokes virtually everywhere, it would be nicer to make the
> > > "tracker" command a shallow interface to subcommands, that are
> > > installed by either tracker core, miner-fs, or whatever.
> >
> > I spotted that;
> >
> > > In the future, we would want the ontology out of Tracker core, but
> > > probably at the time we can claim tracker-store is a generic SPARQL
> > > endpoint :)
> > >
> > > Comments? Maintainers? I won't rush this much, but would certainly
> > > want to tackle this before the branches become too hard to
> > > rebase/merge.
> >
> > I don't really have the desire to become a maintainer, but at the same
> > time it seems like you have to do everything right now so I am willing
> > to step up my involvement a bit... perhaps I could take on one of the
> > repos.
>
> Any help is appreciated :), thanks.

Just a quick reply to say that where the code lives, in this case, is
probably of less importance than how applications can use it.

We need a way to use Tracker within sandboxes, which means getting rid of the
miners running in the session, outside the sandbox, and start having application
controlled mining.

For example, Videos would be able to use the "normal" Tracker 
when outside the sandbox (eg. package installed from the normal distribution),
and have support for the application starting mining within directories it can
access and handle, including mining external drives when requested.

Extend this to Music, Documents, Books, Photos.

The application-controlled mining is probably the most important part of tracker.
If nothing appears on the tracker side, we'll need to have each application
implement their own mining and caching.

Cheers