Re: GConverter commited

[Shaun McCance <shaunm gnome org>]

On Tue, 2009-11-24 at 09:36 +0100, Alexander Larsson wrote:
> On Mon, 2009-11-23 at 21:57 -0600, Shaun McCance wrote:
> 
> > I'll be using the bzip2 and lzma converters in Yelp.  I'm not
> > sure about the magic converter.  I might just throw it away and
> > go off the file name.  The magic detection is not suitable for
> > general use, though I think it's OKish for Yelp.  Although this
> > Debian bug report concerns me:
> > 
> > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=364260
> 
> lzma is detected by shared-mime-info (i.e GContentType), but only if it
> has the right extension. Extending the database with magic info
> shouldn't be that hard though.

Right, so I looked at both shared-mime-info and file when
I was putting together the magic.  The bytes I used came
from file, but they're commented out, because of the bug
I linked to.  See comment #62.

Coincidentally, I *just* received a bug report to support
the new xv in Yelp, which is replacing lzma.  I assume the
new format produced by this will be the one mentioned in
that bug, which does have a sniffable signature.

> > Also, when putting together chains of streams when content may
> > or may not be encoded, I think it would be useful to have some
> > sort of NullConverter, i.e. a GConverter that doesn't change
> > anything.  Would anybody else find that useful?
> 
> I'm not sure why you'd need this, you just don't add a
> GConverterInputStream in that case.

Well, the way I did the magic was with a GConverter that
inspects its data the first time convert is called, and
then constructs another GConverter to pass everything
off to.  (This assumes the first call to convert has at
least some number of bytes, by the way.)

So with my implementation, by the time you get to the
magic, you've already set up a GConverterInputStream
with the magic decompressor.  If the stream turns out
to be uncompressed, you'd have to do a null conversion.
I suppose the magic decompressor could just have the
null conversion built in.

> > Anyway, since I was one of the people wanting this, I thought
> > I'd share my first experiences with it.  I'm curious what other
> > people would like to do about GConverters for other compression
> > schemes.  The code is simple enough that I don't really mind
> > keeping it in Yelp.  But if other people are doing this stuff,
> > maybe we should talk about how to share code.
> 
> Yeah, sharing things like this is good, but we don't want every app to
> link to these libraries, and even gio plugins are not free (even when
> not used) since we load them once to see what extension points they
> support.
> 
> Its not a lot of code either, nor is it very complicated, so maybe cut
> and paste is not such a horrible idea.

I don't disagree.  Is there a place to plop this code
for people to pick it up?  I'll be putting at least the
bz2 and lzma converters into Yelp soon (or possibly an
xv converter instead of the lzmadec-based one).  I'm
still undecided on the magic converter.

--
Shaun