[Tracker] REVIEW: external-crawler branch

[Martyn Russell <martyn lanedo com>]

Hello all,

I've recently been involved in a project to make Tracker the main 
indexer and search engine for content on an embedded platform.

Part of that project is about making remote content available in 
Tracker, for example: 3rd party online cloud based content.

This already presents some interesting problems for Tracker, because the 
only way to handle 3rd party content (right now) is to build an entire 
miner yourself, skipping a lot of the useful logic and filtering work 
that Carlos and the team have added over the years. Specifically, I am 
talking about TrackerIndexingTree, TrackerFileNotifier, 
TrackerFileSystem, TrackerCrawler and TrackerMinerFs classes in 
libtracker-miner.

As a result of this and keeping in mind that we've just released 1.0 and 
are trying to stay stable for users, I've written a new branch trying to 
allow a way to "feed" URIs into Tracker from the bottom and up through 
the stack. That branch is called 'external-crawler':

  https://git.gnome.org/browse/tracker/log/?h=external-crawler

So let me try to outline what this branch does or allows:

1. URIs are no longer restricted to file:// only (yes current master is 
if you want to use most of the libtracker-miner classes).

2. There is a new TrackerEnumerator interface that allows for 3rd 
parties to implement their own URI crawling solution for their backend.

3. A new TrackerFileEnumerator class implementing the TrackerEnumerator 
interface provides the functionality that exists right now. This is 
essentially just calling the GFileEnumerator API as we do in 
master/tracker-1.0 branches right now.

4. Improves existing classes like TrackerIndexingTree, 
TrackerFileNotifier, etc to take a GFile for the root URI, instead of 
always assuming it's file://.

So, please review *looks at Carlos mainly* :) and let me know thoughts 
on the branch.

I would like to merge to master at some point in the next few months in 
time for 1.2.x releases.

Thanks all,

--
Regards,
Martyn

Founder & Director @ Lanedo GmbH.
http://www.linkedin.com/in/martynrussell