Re: [Tracker] Mining GVfs metadata

[Tomas Bzatek <tbzatek redhat com>]

On Fri, 2010-11-19 at 19:05 +0200, Ivan Frade wrote:
> UI information shouldn't be in tracker; but "emblems" can be
> considered tags and the "last played position" is a good property for
> a media file (you could continue playing with a different media
> player!). 

Of course not, we would have to filter some keys out.

> Is there a predefined list of "metadata::" keys? or is completely free
> text? The first case would make our life much easier.

Unfortunately not, applications are free to set whatever metadata they
want. By GIO nature, values can be of type string, number or pointer
(not really useful).

That reminds me I'm not sure which ontology should metadata be mapped
at. Moreover, do we want to store all metadata (filtered from UI stuff)
or rather try to map to existing ontologies? I can imagine applications
using Tracker just to quickly find files marked with metadata of an
app's internal format.

> gvfsd-metadata notifying tracker of changes is definitely the way to
> go. Could that daemon provide directly the data? it can save tracker
> the work of reread the attributes and calculate the difference. 

It could, but imagine that something goes wrong on the other side or
Tracker is not running at all. Then we would have to track already sent
changes and resend those missing ones. I find it more safe to extract
data on demand, initiated from Tracker side.

> "Miner" is just "something that push data into tracker-store". They
> are completely independent from the store and can be developed out of
> the tree. The idea is that each distribution (or even user) chooses
> its own combination of miners, some provided by tracker (like
> filesystem), some written by other teams.

I guess miners are separate processes, does Tracker execute them as
necessary or are they supposed to be running all the time? For our case,
i can imagine a miner that does his job and exits.

> In an ideal case, gvfsd-metadata itself would be a "miner",
> translating metadata writing events into tracker metadata. If this is
> not possible, then the miner-fs must read the extended attributes
> (shouldn't have a big impact in the code). I wonder how this is
> combined with inotify.

I'm now looking at available miners which could serve as examples and
point to start from. 

I'd not rely on inotify, only gvfs metadata daemon knows when it's
finished syncing data. Moreover, as long we store our database in user
homedir and that could very well live on NFS, inotify wouldn't work
there.

If we theoretically make gvfsd-metadata submitting changes to tracker
directly, how do we cope with the initial phase? I mean going through
existing metadata, filling tracker database for the first time. What is
the preferred way?

> We are also around in IRC (#tracker in GIMPnet).
Great, I'll try to hang there.

Thanks,
-- 
Tomas Bzatek <tbzatek redhat com>