Re: [Tracker] Excessive memory use by tracker-store

[Calvin Walton <calvin walton gmail com>]

On Sun, 2010-05-16 at 17:02 +0200, Philip Van Hoof wrote:
> On Sun, 2010-05-16 at 10:43 -0400, Calvin Walton wrote:
> > Well, tracker-store appears to have been fixed, but now tracker-extract
> > is acting up. Within 20 minutes of starting indexing, it was up to 16GB
> > virtual (10GB RSS) memory usage) and was swapping out all my apps, I had
> > to kill it to get my desktop responsive again.
>
> OK, tracker-extract is not as bad as that process can be shut down
> periodically, allowing the kernel to free up all resources each time.
>
> Of course ain't tracker-extract being shut down periodically yet.
>
> However, the proper fix is of course to fix the issue.
> It's likely that there's a serious memory leak or serious memory
> fragmentation in one of the extractor modules.
>
> To find which one and where the leak is, it would be very useful if
> you'd do something along the lines of this:

Thanks for the detailed instructions on using valgrind with tracker.
I'll try to get that run either later this evening or sometime during
next week.

It looks like it is a problem with specifically one of the extractor
modules, as once it got past a certain point in the indexing, I'm not
seeing the memory usage continue to increase.

The indexing's now almost complete; a vast improvement over the older
tracker branch. (tracker-store's only reached ~60mb RSS now, at 84%
indexed)

> > Any new ideas?
>
> Once you have reached the excessive memory usage (hard to know, because
> valgrind adds a significant overhead itself too), CTRL+C Terminal 2 and
> look for a logfile. Compress this logfile and make it available for us
> somehow (note, IRC channel is #tracker on GimpNET).

My IRC nick is 'kepstin', see you around.

-- 
Calvin Walton <calvin walton gmail com>