Re: [Tracker] From the client side

["Marko Anastasov" <marko anastasov gmail com>]

On Sun, Nov 23, 2008 at 3:26 PM, Mikkel Kamstrup Erlandsen
<mikkel kamstrup gmail com> wrote:
> 2008/11/21 Marko Anastasov <marko anastasov gmail com>:
> > Hey all,
> >
> > A few questions:
> >
> > In practice, pulling every single document's (whatever type) metadata
> > from Tracker over D-Bus is slow. For example it takes 10+s on my 2 yr
> > old machine to gather information for about ~1000 documents.
> > How do you see this problem?
>
> Why is it that you need to do that? I mean in practice one should
> never keep more than a "sliding window + cache" in memory on the
> client side. Fx. the 15 visible documents + 30 backwards and forwards
> for fast scrolling.
>
> > I really do not want to see that every time Paperbox starts any more,
> > so I think that I'm going to add a private SQLite database for caching
> > document information so that all documents can be displayed instantly,
> > while querying Tracker for updates will begin in the background.
>
> Before you open that can of worms I think we need to figure what the
> problem really is. Maybe it is with your angle of attack or maybe
> Tracker's API needs a tweak... Could you describe your problem more
> precisely?

The problem is that I think it's impossible to assume what document(s)
the user has in mind when launching the application - is it most
recently opened, recently made/downloaded, or whatever specific;
or maybe she wants to see all stuff with a certain tag. So the only way
to satisfy this is to make all of them available without a significant delay.

While writing this I realized that perhaps I've been making a mistake
by considering a document ready to be displayed only when there
is complete metadata information. I could try rewriting Paperbox in
a way that at first it displays the file names only, then launching the
metadata retrieval in the background to fill in the tiles, with the
"sliding window + cache" approach that you mentioned.

Adding an SQLite database is opening a can of worms indeed.
My question about who should write the metadata to files when
possible (and if that should be done at all) is related to that.

For example, I want to make use of the Google Book API, which returns
some information such as author, description etc [0]. Also I personally
consider note and source URL metadata fields (from DublinCore) very
important let's say. So my question is how can we store this and who
should do it?

Writing to files may work sometimes (PoDoFo [1] looks promising for PDFs;
I don't know if there is a library for writing metadata to Open Document files),
but not always (MS Office, CHM files). But if the metadata is never written to
the file then the metadata exists on only one desktop.

> > I think that there must be a way to sync the metadata database
> > between desktops. It's just not good enough if copying a file on another
> > computer leaves you with blank metadata. I only have tags in my mind
> > now actually but what if Tracker actually stored other useful information
> > such as notes and reference URLs. So maybe an additional tool could
> > export the metadata database contents to some file, and on the other
> > desktop the tool would import the data for recognised files?
>
> Have you seen: http://live.gnome.org/MetadataOnRemovableDevices ?

No, I've read it now. It doesn't seem like the solution to the problem that
I mention because it assumes the presence of removable media which acts
as a carrier of metadata. If I tag a photo and the tags go in Tracker's db,
then copy it to my friend's computer and remove the USB flash immediately
(there's a possibility of using a read-only CD or DVD, but USBs are used much
more frequently these days), then she won't see those tags... *unless* the
daemon recognises metadata.ttl and copies the information into her desktop's
metadata store - is this the plan?

On the other hand, if an application or Tracker itself wrote the tags
to the file
as XMP defines it, then there wouldn't be such a problem, if the application
and/or Tracker decide that they should read these fields that is.

    Marko

[0] http://code.google.com/apis/books/docs/gdata/developers_guide_protocol.html#SearchResultFeed
[1] http://podofo.sourceforge.net/