Re: [Tracker] Profiling tracker

[Carlos Garnacho <carlos lanedo com>]

Hey,

On vie, 2012-01-20 at 14:58 +0530, John stacy wrote:
> Hello Team,
>
> I am trying to profile the latest version of tracker (0.10.37) on my
> ubuntu machine. 

You should try master :), the miner received some improvements there wrt
non-initial indexing there
> My main goal of this analysis are:-
>
> 1. To find out the tracker indexing performance ( crawling time,
> monitors setup time , time consumed by the tracker-store while
> constructing/flushing the database , time taken by the tracker-extract
> to fetch the metadata from a single mp3 file ) 

You'll have to instrument these separately...

> 2. Amount of time spent on the ipc transactions i.e DBUS (between the
> tracker  miner-fs <--> extractor <--> store )

It looks like bustle may help here,
http://blogs.gnome.org/wjjt/2012/01/18/bustle-0-4-0-push-button-receive-d-bus-traffic/

> 3. Current architectural dependencies between the miner/extract/store
> ( serial execution or parallel processing ?)

That's more of a reading exercise. Generally, parallel processing is
done where possible, with the exception of tracker-store insertions
(they happen in a single thread), and miner-fs crawling (the resulting
files are then processed in parallel)

> Can any one suggest some tools/ idea's to achieve the same? 
>
> Initially I thought of putting my own timers at several places of the
> tracker, but since I am not well versed with the tracker code, It may
> not be that useful. 
> Now I am trying to profile using the gprof. 

sysprof is also really handy, although it requires you to compile
everything you want to profile with -fno-omit-frame-pointer

> Can somebody throw some light on the time profiling of tracker?  

With an eye on tracker-miner-fs, during first indexing, most time
consuming tasks are roughly:
1.1) filesystem I/O (tracker-extract)
1.2) dbus/pipe communication (miner-fs <-> tracker-extract)

During non-first indexing (given no new files are indexed):
2.1) Adding directory monitors (this also happens above, but is
negligible compared to these)
2.2) mtime checks (in master this is much improved)

Given 1.1 is really hard to reduce, the most worthwhile thing to knock
down is 1.2, possibly by packing together requests, with some issues to
oversee like handling just added/deleted files between miner-fs doing
checks and tracker-extract acting.

 Carlos
> Thanks In advance,
> John
>
>
>
>
> _______________________________________________
> tracker-list mailing list
> tracker-list gnome org
> http://mail.gnome.org/mailman/listinfo/tracker-list