Re: on crawling upnp contentdirectories...

[Jussi Kukkonen <jku linux intel com>]

On 01/25/2011 01:11 PM, Iago Toral wrote:
> 
> On Tue, 25 Jan 2011 12:12:37 +0200, Jussi Kukkonen <jku linux intel com>
> wrote:
>> Hi all,
>>
>> I've been experimenting with crawling UPnP contentdirectories
>> (mediaservers) lately. For those who haven't seen earlier discussion,
>> the rationale for doing this is that many meadiaservers do not support
>> search, and the browse trees they offer can be pretty weird -- or at
>> least very different from the UI design of a specific media player UI.
>> Crawling and caching would allow us to implement very fast searching
>> locally, avoiding that problem.
>>
>> I've got a grilo-upnp implementation of this (see crawler branch):
>>   http://gitorious.org/jku-grilo/grilo-plugins
>>   git://gitorious.org/jku-grilo/grilo-plugins.git
>>
>> ...but after some discussions we decided I shouldn't send the patches or
>> continue the work, at least for now. Reasons for this:
>>  * caching idea is orthogonal to what grilo currently is
>>  * there are performance issues -- like memory use -- that could be
>> problematic on mobile/embedded scenarios. Grilo just isn't designed for
>> holding unknown numbers of media items in memory "just in case"
>>  * we should be able to do better than per-application, per-instance,
>> non-persistent caching...
> 
> did you consider disk-caching using a lightway database (something like
> sqlite)? I think that would address at least the 2 last bullets.

Briefly, yes. It's just that it feels wrong to implement something that
other components seem to have been designed to do... I could be wrong
about this of course:,I'm just looking at tracker more closely to see if
it is a better fit.

>> So, I'm currently looking at implementing this feature in a better place
>> in the infrastructure. Tracker seems like an obvious choice, even though
>> that probably means quite a bit more work than working with your nice
>> little upnp plugin.
> 
> Mmm... So your current approach is to have tracker using grilo to get
> the crawl the upnp servers and store it in its local database?
> And then, when selecting a upnp server for a search operation, use the
> tracker plugin instead to retrieve the content?

Well, I'm a relative newbie to tracker so I can't say exactly how that
would work yet. My first impression is that there's no reason to use
grilo in tracker. There should probably be a tracker upnp miner that
works similarly to how the fs-miner handles removable media: cache
things for a period of days and only show search results when the source
is actually available (your plugin could be useful as a starting point
for that though).

As to what makes sense for an application... I guess using grilo as the
aggregator makes sense even if data from local disk, removable media and
upnp all come from tracker: the benefit of getting web service data over
same api is still there.


 - Jussi