Re: [Tracker] questions

[Jamie McCracken <jamiemcc blueyonder co uk>]

Samuel Cormier-Iijima wrote:
> On 9/14/06, Jamie McCracken <jamiemcc blueyonder co uk> wrote:
> > Samuel Cormier-Iijima wrote:
> > > I had a couple of quick questions and comments about Tracker:
> > >
> > > How come Tracker uses the low-level DBus interface? Exporting a
> > > GObject over DBus would be *much* simpler (I've tried it :-),
> > > especially for building parameters (GHashTable -> Python dict) and
> > > doing signals (GObject signals get mapped to DBus signals) for the
> > > eventual live query support. Also, this would automagically give us
> > > certain nice things like introspection for free...
> >
> > Do you mean use the dbus binding tool to generate server side gobjects?
> >
> > If so it wont work because tracker is multi-threaded and dbus requests
> > need to be queued for execution by one or more threads (currently only
> > one in the code but its designed for supporting more).
>
> You can use dbus_setup_connection_with_g_main to have DBus requests be
> processed from a specific GMainContext, and then use that from
> whatever thread you want...
>
> It just seems like it would be a lot less maintenance work, but then
> again, "if it ain't broke, don't fix it" :-)

Our dbus connection uses the main thread's mainloop where we place 
incoming request on an async queue which can be serviced by multiple 
threads from a threadpool.

When tracker becomes popular and is used by more third party apps  we 
will need more than one thread here to handle concurrent requests in a 
timely manner.

I am not sure how what you say can help here but feel free to prove 
otherwise. (you woulkd only have one connection/thread with your 
suggestion?)


> > > Also, the DBus interface seems redundant at times, especially when it
> > > asks for a pair of service and uri. I might be missing something here
> > > (i.e. correct me if I'm wrong), but shouldn't the uri be unique? Why
> > > do we have to pass a service type along with it?
> >
> > Because when we support emails they will be in a different db or table
> > so I will need to use the service name to work out which DB or table to
> > use in queries. Storing all services in one table would cause
> > performance problems if email count is very large.
>
> ah, good point, although i thought databases could handle pretty large
> datasets (are many people going to be indexing more than 50,000 files?
> it might get slow even at that number though...)

its more a precaution.

The Dbs are scalabale up to a million rows or so but some people have 
gigabytes of emails and each metadata for an email takes up one row so 
one email  might be 10 rows or so (a file is about the same too although 
music files will have more about 20 or so)

> ok, will see what I can do. if i replaced the DBus calls with
> GObjects, what are the chances of the patch being accepted? :-)

only if it allows multiple threads as I said above to work on *all* 
incoming requests and simplifies the code significantly. I generally 
dont like the overhead of objects and like to keep tracker procedural in 
nature for max speed so such objects need to be lightweight.

 also,
> on that note, like you said, it might be a good idea to get some unit
> tests into tracker. i've been coding up some "stress tests" python
> scripts, and can send them if you'd like.

yes please that would help more than coding the above in gobjects.

> cheers,
> samuel


--
Mr Jamie McCracken
http://jamiemcc.livejournal.com/