Re: [Evolution-hackers] compression for folders?

[Jeffrey Stedfast <fejj ximian com>]

On Mon, 2004-05-10 at 19:02, Ray Lee wrote:
> On Mon, 2004-05-10 at 15:28, Jeffrey Stedfast wrote:
> > you are forgetting the fact that folders are generally not read-only,
> > and so in order to write any new data to the gzip file, you'd have to
> > rewrite it from scratch which negates any speed improvements you could
> > possibly claim.
> 
> ray:~$ echo hello | gzip >test.gz
> ray:~$ echo world | gzip >>test.gz
> ray:~$ zcat test.gz
> hello
> world
> ray:~$

clearly zcat is doing something funky because gzip files have an 8byte
footer that would foul up any code that tried to read that file if it
were mbox.

> 
> As long as the archive folders only support appending, there's no need
> to rewrite the entire file.

it's not trivial to do what zcat does. besides, note "archive". yes,
compressing archive folders was never the issue, the issue was
compressing all folders.

also note that updating flags in messages requires changing headers,
which would mean a rewrite of the gzipped mbox file.

>  Further, there's no need to even keep it in
> one big file (and many good reasons not to). Partition the archives by
> month, or something.
> 
> > also, as a curiosity, I actually tested this theory and it doesn't hold
> > true. reading/inflating a gzip file off disk is no faster than reading
> > the non-compressed file off disk, *and* inflating the gzip file pegs the
> > cpu so if the app was doing other things then it would negatively impact
> > performance of those other operations.
> 
> This rather obviously depends on CPU speed versus disk speed, yes? If I
> had a modern CPU with a device that had a transfer speed of 1 byte a
> second, compressing the stream is an obvious win. If I have a device
> with a transfer speed of 1 GB/s, it's an obvious loss.

right. I did the test on modern hardware. no gains from a gzip file and
a number of obvious downsides.

> 
> As with anything in ComSci, it's a tradeoff.

right, which is why it's important to benchmark it to find whether or
not it's worth doing.

let it be known, that for archive folders - I think it'd be fine, but
definitely not for typical folders.

> 
> I think having some sort of support for an archive system is probably a
> good thing. But the feature has to be balanced by what the team has time
> to support, and from my lurking it sounds like they're already a bit
> overworked.

indeed.

Jeff