Re: [Tracker] miner-fs: Placing monitors on directories takes way too much time

So, I created a new branch for the issue: miner-fs-merge-updates

I couldn't use your patch, as it was not aligned with master, so I
rewrote it with some additional things, you can check it here:

Tried it in several scenarios, and saw a real improvement in 
the initial
crawling and indexing (from ~40s to ~28.5s), around 25% faster.

Thank you for trying it. I just tried your branched code. 
Crawling and indexing time reduced from ~232s to ~108s, more than 50% faster.  

One of the drawbacks seems to be that if the merged sparql 
update fails,
several inserts for different files will be failing, instead 
of just for
one file. We're discussing the issue already, and will try to find the
best solution.

When the merged sparql fails, we'd find a chance to update them one by one. 


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]