Re: [Tracker] RFC: UPnP mediaserver (ContentDirectory) miner

[<ivan frade nokia com>]

Hi!

 Our usual concern with this is the amount of remote content that you need to crawl/cache each time. A very 
huge Upnp server could flood your device with useless data. The canonical example is that you don't want to 
cache the contacts in the LDAP directory of your company. But well, having a miner for it and the option to 
install/run it is not bad at all. I guess the user could even configure it to be active only on his home 
connection. BTW, "Rygel" works the other way around: takes the content from tracker and *exposes* it to other 
UPnp devices. They are complementary.

 Not sure where to put the code. Sounds nice to have all the miners together, it is easy to check the impacts 
of changes in libtracker-miner . But on the other hand miners could have their own independent release 
cycle... do we need a whole release of tracker because we fixed certain miner? and what is better for 
distributions? 

 Regardless where is it, we will review the code and comment on it. Thanks! Really great to see people 
contributing miners!

 Regards,

Ivan
 

________________________________________
From: tracker-list-bounces gnome org [tracker-list-bounces gnome org] on behalf of ext Jussi Kukkonen [jku 
linux intel com]
Sent: 10 February 2011 14:37
To: tracker-list gnome org
Subject: [Tracker] RFC: UPnP mediaserver (ContentDirectory) miner

Hi all,

I'm writing a UPnP ContentDirectory (mediaserver) miner for tracker. See
code at
   git://gitorious.org/tracker-upnp/tracker-upnp.git
   http://gitorious.org/tracker-upnp/tracker-upnp
I'm currently working on it as a independently installable component (as
I need it for tracker 0.10 in MeeGo and assumed I'm way too late to get
it in the release). It would still be nice to get this into tracker git
(better sooner than later), which is why I'm writing this email. Let me
know if you have comments on the idea / implementation.


 * What does it do?

It indexes mediaservers in the local network for media, and stores the
metadata in tracker for mediaplayers to use. Indexing only happens when
there have been changes on the mediaserver. The data is marked
"tracker:available" when the mediaserver in question is available, very
much like data on removable disks. The data should also get removed
after $LIMIT days of not seeing the mediaserver, but this isn't
implemented yet.


 * Does this belong in tracker?

One could argue that indexing media on the network isn't in trackers
scope but I think mediaservers in the local network are just as personal
as removable drives: sometimes you may end up indexing someone elses
data that you won't ever see again, but the data you most often see
really is _your_ data and you will want it to be searchable.


 * Why cache UPnP media metadata at all?

If all UPnP ContentDirectories supported searching there would be no
need to cache the data. Unfortunately there's only one guaranteed way to
access the media: browsing through the "directory" structure imposed by
the media server. There are a couple of problems with the user
experience here:
 1. It is incompatible with just about every media player I've ever seen
-- the players all want to decide themselves how to present the media to
user.
 2. It is quite unscalable: My home network currently has three
mediaservers (music server, laptop, occasionally phone). You can imagine
how much fun it is to look for a song in three totally different
directory structures without any way to search for it...

The only solution is to crawl the directory structures of the less
capable mediaservers and cache the metadata. And if we do that, it makes
sense to index all mediaservers for completeness (but we can use
searching instead of crawling the directories on the servers that
support searching).


* Any drawbacks to this idea?

Quite possibly. UPnP mediaservers have a way of surprising one (mostly
in a negative way). There could be problems with duplicates that are
very difficult to identify (Mediatomb seems to be very good at this),
endless directory recursion, non-standard change notification etc. So
far the positives seem to outweigh the negatives.


Comments welcome,
 Jussi


--
Jussi Kukkonen
Intel Open Source Technology Center

_______________________________________________
tracker-list mailing list
tracker-list gnome org
http://mail.gnome.org/mailman/listinfo/tracker-list