Re: [Tracker] miner-fs: Placing monitors on directories takes way too much time

Hi hi,

What do you think about the idea "grouping the updates for files of one dir into one update"? 

Seems a good idea :-)
Trying your patch right now! Thanks

So, I created a new branch for the issue: miner-fs-merge-updates

I couldn't use your patch, as it was not aligned with master, so I
rewrote it with some additional things, you can check it here:

Tried it in several scenarios, and saw a real improvement in the initial
crawling and indexing (from ~40s to ~28.5s), around 25% faster.

One of the drawbacks seems to be that if the merged sparql update fails,
several inserts for different files will be failing, instead of just for
one file. We're discussing the issue already, and will try to find the
best solution.

Cheers and thanks for the patch!

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]