Re: [xml] Patch to fix ICU flush and pivot buffer

[Joel Hockey <joelhockey chromium org>]

I've updated the patch using git format-patch.

* reverted public interface xmlCharEncInFunc.  It now calls xmlEncInputChunk with flush=1 on all calls as suggested.

Always setting flush=TRUE here makes sense since this is a one-shot conversion of the entire buffer.

* Moved the pivot buf reset to only happen on success (suggestion from Markus).

* removed test/icu_parse_test.xml from patch.  This file is attached separately to this email.

The original bug found by fuzzer only relates to UTF8 decoding, so using Shift-JIS or anything else wont help.  I can't think of any way that this test can simultaneously

 - be effective to test the bug when libxml is compiled with icu

 - still parse correctly and pass when libxml is compiled with iconv

The trick is that we need an encoding which will:

 - not be handled natively by libxml

 - be recognized by icu as UTF8

 - be recognized by iconv as UTF8

On Thu, Oct 26, 2017 at 3:21 AM, Nick Wellnhofer <wellnhofer aevum de> wrote:

On 25/10/2017 17:40, Markus Scherer wrote:

On Wed, Oct 25, 2017 at 4:02 AM, Nick Wellnhofer <wellnhofer aevum de     The patch changes public function xmlCharEncInFunc but this function isn't
    used internally anymore (since commit a78d8036 from 2012). It might still
    be used in client code that wants to use libxml2's character conversion
    facilities, though. Maybe it's better to remove the `flush` parameter and
    always call xmlEncInputChunk with `flush` set to 1. This should at least
    allow one-shot character conversions without breaking the public API.

The ICU conversion functions must be called with flush=FALSE before the end of the stream, and with flush=TRUE at the end of the stream.

Yes, but I'm only talking about xmlCharEncInFunc which isn't used internally.

Nick

Attachment: 0001-Fixed-ICU-to-set-flush-correctly-and-provide-pivot-b.patch
Description: Text Data

Attachment: icu_parse_test.xml
Description: XML document