Re: [Tracker] miner-fs: Placing monitors on directories takes way too much time

On 03/09/10 03:10, Chen, Zhenqiang wrote:
But my test shows tracker related modules take ~150% CPU, especially

Test HW: ASPIREone (netbook)  SW: MeeGo, tracker-0.9.16, sqlite-3.6.

tracker-control -r tracker-control -s top

You can use tracker-control -rs btw ;)

tracker-store: ~95% dbus-daemon:   ~22% tracker-miner-fs: ~18%
tracker-extractor: ~12%

The store is expected to take almost ALL the time here as it does the most IO operations.

That sounds high for d-bus, are you using d-bus >= 1.3.1? That should improve performance here.

As I understand most time consuming work are IO opertions, which
should not take much CPU. As comparison, I cp all the files, it takes
only ~30% CPU.

Sure, but there's a difference between copying data which is quite a straight forward operation and extracting data and putting it into a database where relationships between the data are described. Not to mention the redundancy we add in case there is corruption, etc, etc.

The data for dbus-daemon, tracker-miner-fs and tracker-extractor seam
reasonable. Do you think it is reasonable for tracker-store?

Totally. It has the most work to do.

Profiles show "sqlite3_step" indirectly called in "update_sparql"
takes most of the time.

I would expect that.

Any comments on how to improve it?

Are you using glib unicode parsing? We changed configure to require !glib yesterday, the indexing performance difference is incredible, solved with:

  sudo apt-get install libunistring-dev

Upgrading to D-Bus 3.7 and getting the latest 0.9.x release will make quite a difference too (with direct-access in there too the miner-fs communication is faster) and signals emitted are more efficient too since yesterday's release.

Thanks! -Zhenqiang

No problem.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]