Re: [Tracker] miner-fs: Placing monitors on directories takes way too much time

Le Thu, 9 Sep 2010 17:04:06 +0800,
"Chen, Zhenqiang" <zhenqiang chen intel com> a écrit :

I will try d-bus-1.3 and latest tracker.

Opps, yea, I meant 1.3.1, not 3.7. Also, note, 1.3.0 is buggy so you 
will need 1.3.1. This should avoid quite some memory copies 
when indexing.

I tried tracker git code (master, last update Sep 7). But test
results show it is ~15% slower than tracker-0.9.16. And I tried
d-bus-1.3.1. It did not help in my case. 

dbus is one of the bottlenecks. Tests show 1000 continuous INSERTs
will block the d-bus in my system. 
If you use DBus 1.3.1 or higher (1.3 is broken), you'll benefit from a
new update mechanism that greatly offloads the load on DBus, and makes
things much (much) faster. And if you combine that with SQLite unstable
(3.7+), you also get direct access, which makes allows a new query
mechanism that bypasses DBus totally.

To reduce the dbus overhead, I tried to group the UPDATE for files of
a dir into one update. Tests show it is ~2X faster. Here are the
logs: (Notes: among the 10693 files, 10654 files are photos.) 

tracker git master:
Finished mining in seconds:232.713859, total directories:354, total

Finished mining in seconds:200.689239, total directories:354, total

tracker-0.9.16 with group update (testing code segment is attached.)
Finished mining in seconds:92.813985, total directories:354, total

What do you think about the idea "grouping the updates for files of
one dir into one update"? 


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]