Re: [Tracker] Clues regarding improving performance of tracker-store

[Adrien Bustany <abustany gnome org>]

Le 2013-07-12 15:45, Jonatan Pålsson a écrit :
> Hi everyone,
>
> I'm interested in a general speedup of indexing in Tracker, and have
> therefore been running some performance profiling on tracker-extract,
> tracker-miner-fs and tracker-store using OProfile. I found that a lot
> of time is spent in the store compared to the other processes (in
> rough numbers, it is along the lines of 70% time in store, 15% each
> for the extractor and miner) according to OProfile. I was indexing MP3
> files when I came up with these numbers.
>
> In order to speed things up, I tried modifying the PRAGMA statements
> sent to SQLite upon database initialization, but with no real success.
>
> I'm looking for a discussion of what I can try in order to increase
> performance of the store. For me, it seems logical that smaller /
> simpler ontologies would yield a performance increase in scenarios
> where it is possible to use such ontologies - for instance removing
> ontologies related to E-mail when the system running Tracker does not
> have E-mail capabilities.
>
> Do you think slimming down the ontologies could yield a performance
> boost for the store?
>
> Is there support for altering the ontologies? I have seen
> .ontology-files. It seems to me that these are the place to go.
>
> The wording of [1] seems to indicate that there are more places in
> Tracker where ontologies can be modified other than the .ontology
> files. Am I reading this correctly, or are all ontology
> related-matters handled through the .ontology files?
>
> Finally, do you have any (possibly vague) clues as to what I can look
> at in the store to improve its performance? Any suggestions are
> welcome!
>
> Have a nice weekend everyone!
>
> [1] 
> https://wiki.gnome.org/Tracker/Documentation/SupportedOntologyChanges
>
> --
> Regards,
> Jonatan Pålsson
>
> Pelagicore AB
> Ekelundsgatan 4, 6th floor, SE-411 18 Gothenburg, Sweden

Hi,

one factor is the "depth" of ontologies, ie how many subclasses you have 
for the resources that actually interest you. If you insert a resource 
of type X, and X has 5 superclasses, then you get 5 inserts in SQLite. 
Matthias has been analyzing this in 
http://taschenorakel.de/mathias/2012/04/25/fulltext-search-datastore/ 
(and the previous posts on the same subject, linked from this one). Of 
course, "fixing" that would be an incompatible change with the standard 
ontologies.

Cheers

Adrien