Re: [Tracker] REVIEW: external-crawler branch

[Sam Thursfield <ssssam gmail com>]

hi Martyn!

I reviewed the branch before comprehending your overview of what it does, so there are some comments on patches that are later completely overwritten. I've left them in case they are helpful in some way. It'd be nice to squash a bunch of the commits in the branch together though before merging, if you get time, so the history is clearer!

I've attached some actual comments, as a text file because I imagine gmail will mangle it otherwise (I should really stop using gmail :)

Sam

On Fri, Jun 6, 2014 at 3:16 PM, Martyn Russell <martyn lanedo com> wrote:

Hello all,

I've recently been involved in a project to make Tracker the main indexer and search engine for content on an embedded platform.

Part of that project is about making remote content available in Tracker, for example: 3rd party online cloud based content.

This already presents some interesting problems for Tracker, because the only way to handle 3rd party content (right now) is to build an entire miner yourself, skipping a lot of the useful logic and filtering work that Carlos and the team have added over the years. Specifically, I am talking about TrackerIndexingTree, TrackerFileNotifier, TrackerFileSystem, TrackerCrawler and TrackerMinerFs classes in libtracker-miner.

As a result of this and keeping in mind that we've just released 1.0 and are trying to stay stable for users, I've written a new branch trying to allow a way to "feed" URIs into Tracker from the bottom and up through the stack. That branch is called 'external-crawler':

  https://git.gnome.org/browse/tracker/log/?h=external-crawler

So let me try to outline what this branch does or allows:

1. URIs are no longer restricted to file:// only (yes current master is if you want to use most of the libtracker-miner classes).

2. There is a new TrackerEnumerator interface that allows for 3rd parties to implement their own URI crawling solution for their backend.

3. A new TrackerFileEnumerator class implementing the TrackerEnumerator interface provides the functionality that exists right now. This is essentially just calling the GFileEnumerator API as we do in master/tracker-1.0 branches right now.

4. Improves existing classes like TrackerIndexingTree, TrackerFileNotifier, etc to take a GFile for the root URI, instead of always assuming it's file://.

So, please review *looks at Carlos mainly* :) and let me know thoughts on the branch.

I would like to merge to master at some point in the next few months in time for 1.2.x releases.

Thanks all,

--
Regards,
Martyn

Founder & Director @ Lanedo GmbH.
http://www.linkedin.com/in/martynrussell
_______________________________________________
tracker-list mailing list
tracker-list gnome org
https://mail.gnome.org/mailman/listinfo/tracker-list

Attachment: tracker-external-crawler-review.txt
Description: Text document