[Tracker] RFC: UPnP mediaserver (ContentDirectory) miner

[Jussi Kukkonen <jku linux intel com>]

Hi all,

I'm writing a UPnP ContentDirectory (mediaserver) miner for tracker. See
code at
   git://gitorious.org/tracker-upnp/tracker-upnp.git
   http://gitorious.org/tracker-upnp/tracker-upnp
I'm currently working on it as a independently installable component (as
I need it for tracker 0.10 in MeeGo and assumed I'm way too late to get
it in the release). It would still be nice to get this into tracker git
(better sooner than later), which is why I'm writing this email. Let me
know if you have comments on the idea / implementation.


 * What does it do?

It indexes mediaservers in the local network for media, and stores the
metadata in tracker for mediaplayers to use. Indexing only happens when
there have been changes on the mediaserver. The data is marked
"tracker:available" when the mediaserver in question is available, very
much like data on removable disks. The data should also get removed
after $LIMIT days of not seeing the mediaserver, but this isn't
implemented yet.


 * Does this belong in tracker?

One could argue that indexing media on the network isn't in trackers
scope but I think mediaservers in the local network are just as personal
as removable drives: sometimes you may end up indexing someone elses
data that you won't ever see again, but the data you most often see
really is _your_ data and you will want it to be searchable.


 * Why cache UPnP media metadata at all?

If all UPnP ContentDirectories supported searching there would be no
need to cache the data. Unfortunately there's only one guaranteed way to
access the media: browsing through the "directory" structure imposed by
the media server. There are a couple of problems with the user
experience here:
 1. It is incompatible with just about every media player I've ever seen
-- the players all want to decide themselves how to present the media to
user.
 2. It is quite unscalable: My home network currently has three
mediaservers (music server, laptop, occasionally phone). You can imagine
how much fun it is to look for a song in three totally different
directory structures without any way to search for it...

The only solution is to crawl the directory structures of the less
capable mediaservers and cache the metadata. And if we do that, it makes
sense to index all mediaservers for completeness (but we can use
searching instead of crawling the directories on the servers that
support searching).


* Any drawbacks to this idea?

Quite possibly. UPnP mediaservers have a way of surprising one (mostly
in a negative way). There could be problems with duplicates that are
very difficult to identify (Mediatomb seems to be very good at this),
endless directory recursion, non-standard change notification etc. So
far the positives seem to outweigh the negatives.


Comments welcome,
 Jussi


-- 
Jussi Kukkonen
Intel Open Source Technology Center