Questions on DBus / Mediaserver2 interface and very large media libraries

[James Wyper <jrwyper yahoo co uk>]

Hello

I've written a DBus MediaServer2 server for Rygel; the prime motivation 
for this was to have a backend that allowed more flexibility in how my 
music media was organised into containers (e.g. using the Composer as 
well as Artist tag).

The code is at https://github.com/jameswyper/avice but before you rush 
to take a look be aware that

- it's in Ruby so will be (relatively) slow and seems to consume a lot 
of memory (about 6k per DBus Object)
- the code is in pre-alpha state, it's just a collection of editable 
scripts with almost no error-handling
- I need to add the signal for a container's contents changing (doesn't 
matter right now as the scripts don't handle changes to my media library 
yet, so if I start the server then start rygel the signals aren't ever 
required)
- for some reason the tracks are served up in title rather than 
tracknumber sequence - not yet sure if this is my server or a rygel issue


- all that said, it does work for me (I'm listening to the results now!)

With a set of ~1000 tracks the performance appears fine, however with my 
full music collection rygel throws loads of error messages about DBus 
timeouts.  My full collection is about 30,000 tracks and because I put 
each track into multiple virtual containers (by Artist, By Album, By 
Composer etc) I'm creating around 100,000 MediaItem objects.

I haven't yet tested to see at what point (number of tracks) the system 
breaks.

I would expect rygel to only query the objects / containers it needs to, 
e.g when the UPnP client requests the contents of the root folder, rygel 
requests this over DBus, when the client selects the "Artists" 
container, rygel then requests the contents of that container.. and so 
on until the desired tracks are found.  However when I start my server 
and then start rygel, it appears to try and iterate through the whole 
media hierarchy (before I connect a client) - presumably keeping its own 
copy in memory somewhere.  The result is loads of DBus timeouts and 
rygel itself is unresponsive (my UPnP client hangs or times out).

Is this how rygel is designed (keeping its own copy of the media 
hierarchy)?  If it is, then it might not be a practical solution for me 
because of the data volumes involved.  Can someone briefly describe for 
me the logic for rygel issuing MediaServer2 calls over DBus?

Would changing the DBus timeout parameter (not sure where) help?

Does anyone else have experience using rygel (with whatever backend) 
with media libraries of this size?  Although I have no vala coding 
experience I might switch tack and have a go at hacking the tracker or 
mediaexport plugin to allow for a configurable container hierarchy if 
that's more likely to work..

Apologies, lots of rambling questions here, but would welcome your thoughts

Thanks
James