Re: [xml] strange encoding behavior when parsing HTML files

[Aaron Patterson <aaron patterson gmail com>]

On Fri, Apr 17, 2009 at 1:53 AM, Daniel Veillard <veillard redhat com> wrote:
> On Thu, Apr 16, 2009 at 01:51:10PM -0700, Aaron Patterson wrote:
> > Hi,
> >
> > There seems to be strange behavior in libxml2 with regard to encoding
> > when parsing an HTML file.  If an HTML file contains a meta tag
> > hinting at the encoding, libxml2 will use the encoding in the meta tag
> > *unless* there are strange characters before the meta tag.
> >
> > If there are strange characters before the meta tag, libxml2 will
> > guess the encoding and use the guessed encoding for the rest of the
> > document even though the meta tag reported the correct encoding.
> > What's worse is that libxml2 will report that it used the encoding
> > from the meta tag when outputting the content of the document
> > indicates that it did not.
> >
> > Here is an example of the behavior in action:
> >
> >   http://gist.github.com/96641
> >
> > fail.html fails, and success.html "does the right thing".
> >
> > Should I report this in bugzilla?
>
>  Yes please. The encoding handling is a real problem in HTML
> because you can get content and hence have to parse before possibly
> getting the meta tag (if available !)
>  That was fixed in XML by the xmlDecl and rules to parse it without
> encoding informations a priori.

Thanks Daniel, I will file a bug.  Also, I would like to be a helpful
bug reporter by at least providing a patch containing a failing test.
Where should I start?  Should I add to testHTML.c?

Thanks!

-- 
Aaron Patterson
http://tenderlovemaking.com/