[Tracker] Trying out ordered deletes (was: Making some of our team-plannig public)

[Carlos Garnacho <carlos lanedo com>]

Hey :),

Philip wrote:
> o. Some item_remove performance improvements:
>
>    Right now for each delete two queries are executed
>      o. Set the tracker:available to false   - (a)
>      o. Delete the actual resource           - (b)
>                                                                  
>       They are put in the process pool with the same priority. (look for
>       tracker_processing_pool_push_ready_task in item_remove). This
>       means that the queue looks like this:
>
>       ababababababababababababab
>
>       The first query (a) is being done because (b) is slow. But looking
>       at the queue it's not helping much: The queue should look like
>       this instead:
>
>       aaaaaaaaaaaaabbbbbbbbbbbbb

I've been trying out this idea, this is now implemented in the
grouped-deletes branch, I've added bulk tasks, which gather a collection
of similar operations, the sparql is composed on flush. Due to event
granularity and intermediate flushes, processing is rather something
like:

   aaabbb aaaabbbb aabb ab aabb ...

which also helps in getting a somewhat prompt response as items are
processed.

> From my timings, it causes little/no effect to small deletes, but seems
to help increasingly for bigger deletes, Deleting a folder containing
141 mp3 took about the same (both ~5.6s) on this laptop, Deleting an
unclean gtk+ repo shows a ~1min reduction (~116s vs ~172s) on the same
machine.

Cheers,
  Carlos