Re: [Tracker] Using setrlimit() is causing a lot of SIGSEGV

[Martyn Russell <martyn imendio com>]

Jamie McCracken wrote:
> On Fri, 2009-04-03 at 15:18 +0100, Martyn Russell wrote:
> > I am not entirely sure setting a memory limit helps. Right now it
> > masks the problem with SIGSEGV and we just get a lot of those. I
> > would much rather someone got their data indexed and then
> > complained about memory usage than having 120 crash reports in a
> > week (as I just have had) mostly down to this limit (as it seems).
> >
> > Consider this. You mmap a file (to get metadata) as we do in some
> > cases, that file is 150Mb, that instantly causes a SIGSEGV with the
> > current solution.
> >
> > The limits right now are 128Mb and 512Mb for AMD64 processors (not
> > sure why we have a difference there).
> >
> > I would like to see this scrapped and to have people report the 
> > extractor when it eats too much memory. Then we can take each file
> > on a case by case basis.
>
>
> I would rather we increase the memory usage permitted rather than
> risk flames of tracker making their system unusable.

We already have this with the current solution. Right now people are 
already saying they get too many crashes and they are disabling indexing 
or removing Tracker because we are over protective.

If we are really sucking up all memory and making the machine unusable 
we really need to fix it, not put a measure in place expecting it.

> As a compromise, perhaps we could make exceptions for some extractors
> which we know always uses mmap and are therefore less likely to cause
> swappage?

Maybe. I would like to hear what others have to say. I would also like 
to see how things work out without any measures for a bit and gauge 
memory consumption over a few weeks. We can do that without a release 
since we are of course testing all the time on the Maemo platform. I 
would suggest we do that and see how things to then decide.

--
Regards,
Martyn