Re: libxml2 commit 65c7d3b2e6506283eecd19a23dcf0122fbcdac33

[Daniel Veillard <veillard redhat com>]

  Hi Colin,

On Sat, Aug 04, 2012 at 04:59:29AM -0400, Colin Walters wrote:
> Hi Daniel,
> 
> It looks like
> http://git.gnome.org/browse/libxml2/commit/?id=65c7d3b2e6506283eecd19a23dcf0122fbcdac33
> 
> will require changes in evolution-data-server at least, and potentially
> other GNOME modules.  While jhbuild is presently stuck on a 2.8.0
> tarball[1], my unofficial build system tracks git.
> 
> Are you planning to make the new libxml2 parallel installable (i.e. bump
> pkg-config version, soname, etc.)?  
> 
> Or should GNOME be tracking a stable branch of libxml2, and git master
> is now a research/future-API version?
> 
> [1] In reality a lot of jhbuilders now use sysdeps to avoid building
> libxml2 at all, but that just defers this issue even farther.

  So I looked at this more closely. It happens that evolution-data-server
was using raw xmlOutputBuffer to serialize XML, and then accessing
directly the fields inside one of the buffer of xmlOutputBuffer. The API
allowed it by exposing everything from the header to public space,
mistake done circa 98-99 IIRC and a bit late to fix ... The problem are
that those buffers were using int instead of size_t for various size
leading to a variety of troubles including security ones. How to fix
that while keeping everything pblic API and ABI compatible ? Not doable
IMHO. So I did change one of the inner buffer structure of the parser
input and output to make them private, and fixed the issue there, but
there is still some application using them like evolution-data-server.
  I don't see it worth maintaining another 'libxml2' in parallel for
this, I still have the scars from libxml/libxml2 (I was able to kill
libxml at the distro level only a few years ago, I don't want to get
into that situation again !).
  The new buffer structure will be ABI compatible with the old ones,
i.e. the old code as compiled wil be able to work with the new one, as
the fields with the same values are in the same place in the new
structures. But the structure are now opaque and the few places where
the code was using it directly will need fixing.
  What I see from the usage there are:

  buf = xmlAllocOutputBuffer (NULL);
  ....dump stuff to the buffer...
  use data at buf->buffer->content, of size buf->buffer->use

First okay, that was allowed by the API, but such buffers were supposed
to be used for I/O and encoding conversion, in general accessing
buf->buffer->content and buf->buffer->use directly was not really the
expected way to do things. The fact that xmlOutputBuffer were not
supposed to be used that way is the reason why there is no accessors for
getting the output data, this is now fixed as of commit

  http://git.gnome.org/browse/libxml2/commit/?id=e258adecd0e19a6cfe6afa232b89aa416368820e

 So where there is such use of direct access, check the LIBXML2_NEW_BUFFER
macro and if present then
   - replace buf->buffer->content with xmlOutputBufferGetContent(buf)
   - replace buf->buffer->use with xmlOutputBufferGetSize(buf)

leading to something along those lines:
--- calendar/backends/caldav/e-cal-backend-caldav.c.orig	2012-08-06 12:39:16.368456121 +0800
+++ calendar/backends/caldav/e-cal-backend-caldav.c	2012-08-06 12:41:20.602442480 +0800
@@ -1792,11 +1792,19 @@ caldav_receive_schedule_outbox_url (ECal
 	soup_message_headers_append (message->request_headers, "User-Agent", "Evolution/" VERSION);
 	soup_message_headers_append (message->request_headers, "Depth", "0");
 
+#ifdef  LIBXML2_NEW_BUFFER
+	soup_message_set_request (message,
+				  "application/xml",
+				  SOUP_MEMORY_COPY,
+				  (gchar *) xmlOutputBufferGetContent(buf),
+				  xmlOutputBufferGetSize(buf));
+#else
 	soup_message_set_request (message,
 				  "application/xml",
 				  SOUP_MEMORY_COPY,
 				  (gchar *) buf->buffer->content,
 				  buf->buffer->use);
+#endif
 
 	/* Send the request now */
 	send_and_handle_redirection (cbdav->priv->session, message, NULL);

  There are other chunks needing identical fixes, and another C file
seems to exhibit that pattern too, I would gladly provide a ful fix,
but I'm unsure yo realy want the patch done that way.

  if in some place the xmlBufferPtr was passed independantly of the
OutputBuffer, it's possible to use xmlBufGetContent(buffer) and
xmlBufUse(buffer) to achieve the same.

 If there are other places where the xmlOutputBuffer and
xmlParserInputBuffer changes in libxml2 git head give problems I'm
ready to help out.

  As I said I don't plan to make an official release with the changes
before September, so there is a bit of time to get this all cleaned up.

Daniel

-- 
Daniel Veillard      | libxml Gnome XML XSLT toolkit  http://xmlsoft.org/
daniel veillard com  | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library  http://libvirt.org/

-- 
Daniel Veillard      | libxml Gnome XML XSLT toolkit  http://xmlsoft.org/
daniel veillard com  | Rpmfind RPM search engine http://rpmfind.net/
http://veillard.com/ | virtualization library  http://libvirt.org/