Re: Beagle CPU usage (was Proposed module: tracker)

[Bastien Nocera <hadess hadess net>]

On Tue, 2007-03-27 at 14:52 -0600, Federico Mena Quintero wrote:
> El mar, 27-03-2007 a las 00:54 +0100, Bastien Nocera escribió:
> > > 
> > > I implemented your code some time ago in Totem's unstable branch, and I
> > > was wondering why movies weren't getting thumbnailed anymore. I couldn't
> > > mmap files larger than 256 megs.
> [snip]
> > FYI, I've modified Totem's code to take into account the input file size
> > when thumbnailing a file, so the mmap won't fail, and I get 256 megs of
> > real memory (hopefully) to process the file.
> > 
> > Other thumbnailers that need to map large files might have different
> > needs (eg. mmap'ing large fonts, or images). The thumbnailers have the
> > knowledge to do that, not the libgnomeui code.
> 
> Woot, I didn't realize that mmap() would bite us.  Maybe RLIMIT_DATA
> would serve us better.
> 
> What happens with mmap-based allocators like GSlice?  I wouldn't want a
> runaway, glib-based thumbnailer to bypass RLIMIT_DATA.  Does anyone know
> what the kernel does?

RLIMIT_DATA might work, althought on some platforms, it limits mmap as
well.

> [/me wonders if thumbnailers are measurably faster by using mmap()
> instead of read()...]

Not that much, but it allows to read only what's necessary off the disk,
and it's the kernel actually doing the work.

I'll note that even if it doesn't do anything, modifying the
thumbnailers to not mmap the files they're trying to thumbnail goes
against your original point of the thumbnailers being third-party
applications we had no control over.

> [Can you have thumbnailable files that are too large to mmap() even with
> a full address space?  8 GB videos on a 32-bit box?]

Fall back to reading by hand.

-- 
Bastien Nocera <hadess hadess net>