Re: [xml] Questions about libxml as a DLL on win32.

[Steven M Cherry <smc cherrys org>]

Hi,

I don't use the 95/98/ME branch of windows either.  It's not viable for
serious server programming. :-)

Here's a very simple demonstration of the dll memory problem.  There are 4
files here.  It's set up for a borland compiler, but this should be very
simple to buid a different makefile for MSVC.  The code for the dll has two
simple functions that wrapper malloc and free.  The code in main.c uses
the dll in 2 loops.  The first loop allocates and frees using the dll 
functions.  The second loop allocates using the dll, but frees using a local
call to free().

Both the dll and the main exe are compiled with the same set of library
flags and compiler options.

The thing to watch is not the application itself.  If you have enough
memory on your system you won't actually see the application crash or
have any problems.  The thing to watch is your task manager display and the
memory useage of the application.

During the first loop, when it is allocating and freeing by calling the
functions in the dll, the memory useage will stay constant.  However, as
soon as the second loop starts, the memory useage will shoot up constantly.
This is because memory allocated in a dll cannot be legally freed in the
main process or another dll.

Makefile.borl
-------------
#
# Simple makefile to create a dll and a main executable.
###############################################################

CC=bcc32
IMPLIB=implib
OHEXT=obj


CFLAGS=-w -VM -WM -VF 
LFLAGS=

.cpp.obj:
        $(CC) $(CFLAGS) -c $<

DOTOH=dllfunctions.$(OHEXT) main.$(OHEXT)

all: $(DOTOH)
        $(CC) $(CFLAGS) -WD -edllfunctions.dll dllfunctions.$(OHEXT)
        $(IMPLIB) dllfunctions dllfunctions.dll
        $(CC) $(CFLAGS) -WC -etestdll.exe main.$(OHEXT) dllfunctions.lib

clean:
        del *.$(OHEXT) 
        del dllfunctions.dll 
        del dllfunctions.lib
        del testdll.exe

main.c
------
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include "dllfunctions.h"

int main (void)
{
        int i;
        char* ptr;

        printf("Allocating and freeing using both functions.\n");

        for(i = 0; i < 2000000; i ++){
                ptr = create_memory(2048);
                memset(ptr, 'A', 2048);
                release_memory(ptr);
        }

        printf("Now use our own copy of free()\n");

        for(i = 0; i < 2000000; i ++){
                ptr = create_memory(2048);
                memset(ptr, 'A', 2048);
                free(ptr);
        }
        return 0;
}               

dllfunctions.h
--------------
#ifndef DLLFUNCTIONS_H
#define DLLFUNCTIONS_H

#ifdef _WIN32
#       ifndef DLLEXPORT
#               define DLLEXPORT __declspec(dllexport)
#       endif
#else
#       define DLLEXPORT
#endif

/**
  * Simple function to allocate memory and then return it to the
  * caller.
  */
DLLEXPORT char* create_memory(size_t len);

/**
  * Simple function to free an allocated memory pointer.  This uses
  * free().
  */
DLLEXPORT void release_memory(void* ptr);

#endif /* DLLFUNCTIONS_H Defined */


dllfunctions.c
--------------
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include "dllfunctions.h"

char* create_memory(size_t len)
{
        char* ptr;
        ptr = (char*)malloc(len);
        if(ptr == NULL){
                printf("Error allocating memory in create_memory\n");
                return NULL;
        } 
        return ptr;
}

void release_memory(void* ptr)
{
        if(ptr != NULL)
                free(ptr);
}


Now, I can't show you where this rule is in official MS documentation,
I had to find it out myself by trial and error.  But as far as running this
on NT and win2k, the results are always consistent.

If you see the same things that I'm seeing, then my proposal is to add
a flag to the libxml build process that indicates when the win32 DLL is
being created.  When this is the case, we should make xmlFree() a regular
function instead of a function pointer.  This will allow memory to be
created inside the libxml dll, and then free'd correctly with the same
call to xmlFree() as everyone else uses.

Thanks,

Steven

> Hello there.
>
> > Basically a DLL has a boundary layer around it that enforces memory
> > allocation and deallocations to happen on the same side of the boundary.
> > Memory can be shared and read and written across this boundary, but
> > if it is allocated on one side of the boundary, it must be freed on the
> > same side of the boundary.
>
> Hm... perhaps in Win95/98/ME, which I neither use nor understand. NT-kernel based
> systems, such as WinNT and Win2k have a linear address space. What you are saying
> is totally new to me. I doubt this is true. Can you point me to the official
> documentation which describes this?
>
> > The function pointer nature of xmlFree ends up violating this memory
> > boundary layer of the DLL.  What ends up happening is that the xmlFree
> > pointer is initialized inside the DLL and therefore points to the deallocation
> > routine that is valid for the DLL.  However, when my code calls xmlFree
> > to free up strings allocated by libxml (xmlGetProp mainly) I get seg
> > faults (the windows equivalent) because my code is not allowed to execute
> > the DLL's memory deallocation routines.
> > So basically libxml has exported a pointer to an operating system protected
> > set of code that is only allowed to be used by the libxml DLL itself.
>
> The function pointer is a variable exported from the library and can be accessed
> by the client application at will. Wether it contains a valid function address or
> not, depends on what client app writes into this variable. The address of the
> function pointer, means the address of the exported variable itself, is in the
> space designated for libxml by the dynamic loader.
>
> Please double check what have you done, for I can neither understand nor
> reproduce the problem here. If you cannot find the clue, please post a snippet of
> code which shows how do you set the value of the exported function pointer (if
> you do this) and how do you call the function through that pointer.
>
> > I use NT 4.0, and Win2000.  I use the Borland 5.5 free compiler to do
> > all of my NT development.  Which also, btw, means that I've ported libxml
> > to the borland 5.5 compiler environment to build the DLL.  If anybody's
> > interested, I can send my diff's to the list.
>
> Nostalgia... Many days have passed since I have had the pleasure to use Borland's
> 5.x compiler. Of course you can send the diffs. If they don't conflict with
> something else, noone shall object on applying them.
>
>
> Cheers
> Igor
>
> ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿñ?Y??X§?X¬·¥þ     è?ïè®m¶?ÿþf¢?ø'¢gÿ¢¸??¨¥?©ÿ?+-?wèÿ¥

-- 
==================================================================
== Steven M. Cherry                            smc cherrys org  ==
==                 http://family.cherrys.org/steven/index.html  ==
==================================================================