Re: [xml] strange encoding behavior when parsing HTML files

[Aaron Patterson <aaron patterson gmail com>]

On Fri, Apr 17, 2009 at 1:53 AM, Daniel Veillard <veillard redhat com> wrote:
> On Thu, Apr 16, 2009 at 01:51:10PM -0700, Aaron Patterson wrote:
> > Hi,
> >
> > There seems to be strange behavior in libxml2 with regard to encoding
> > when parsing an HTML file.  If an HTML file contains a meta tag
> > hinting at the encoding, libxml2 will use the encoding in the meta tag
> > *unless* there are strange characters before the meta tag.
> >
> > If there are strange characters before the meta tag, libxml2 will
> > guess the encoding and use the guessed encoding for the rest of the
> > document even though the meta tag reported the correct encoding.
> > What's worse is that libxml2 will report that it used the encoding
> > from the meta tag when outputting the content of the document
> > indicates that it did not.
> >
> > Here is an example of the behavior in action:
> >
> >   http://gist.github.com/96641
> >
> > fail.html fails, and success.html "does the right thing".
> >
> > Should I report this in bugzilla?
>
>  Yes please. The encoding handling is a real problem in HTML
> because you can get content and hence have to parse before possibly
> getting the meta tag (if available !)
>  That was fixed in XML by the xmlDecl and rules to parse it without
> encoding informations a priori.

I've reported the bug here:

  http://bugzilla.gnome.org/show_bug.cgi?id=579317

I wasn't sure how I should set the priority.  I set it to critical
because my data is incorrect and I don't have a work around besides
parsing the document myself, looking for the encoding, then passing
the encoding to libxml2.

Thanks for the help!

-- 
Aaron Patterson
http://tenderlovemaking.com/