Re: [Tracker] Using setrlimit() is causing a lot of SIGSEGV

[Martyn Russell <martyn imendio com>]

Jamie McCracken wrote:
> On Fri, 2009-04-03 at 12:09 +0100, Martyn Russell wrote:
> > Hi all,
> >
> > I have spent a lot of time this week triaging bug reports in Launchpad 
> > and I noticed ubuntu patches the setrlimit() call we use. I haven't seen 
> > exactly how, but we did some light testing here by not using it at all, 
> > and the files which were causing tracker-extract to crash before, now 
> > are not. The files are, however, being extracted perfectly.
> >
> > I would suggest we remove ALL calls to setrlimit in 
> > trackerd/tracker-indexer/tracker-extract. Certainly the extractor.
> >
> > Does anyone object to this?
>
>
> yes from experience corrupted files can turn the extract process into a
> zombie
>
> Also memory usage can rocket especially with video files
>
> It might be a case we extend the time permitted for the extractor to
> finish so that slower machines are not penalised but I would object to
> removing this safety net

Time really is not the issue here, but the memory limitation is. Plus we 
already have 2 or 3 mechanics in place to make sure extraction doesn't 
take too long.

1. DBus message timeout (10-15 seconds) + kill from indexer
2. alarm()
3. setrlimit()

A lot of bugs and I mean a lot, are being reported due to the memory 
issue not the cpu time issue from what I can see. Carlos' test actually 
backs this up.

--
Regards,
Martyn