[xml] RFC: New build process on Windows/MSVC

[Igor Zlatkovic <igor stud fh-frankfurt de>]

Hello everyone,

few days ago, I have announced a new build process for libxml, an here I
would like to introduce it and collect your opinions.

To give a quick overview, here is what happens:

* The new build process does not use any IDE. It uses a plain text Makefile,
targeting Microsoft's 'nmake' utility and Microsoft's compiler suite.
* It adds a configuration stage to the build process.
* It adds an installation stage to the build process.
* It does not interferre with the current build process.
* It contains only files in text format. Line-end style is irrelevant. The
files have well-known makefile and Javascript syntax, so they can be put
into the CVS in text mode and patched just like any other source files.

How does it work? Well, typically, one would unpack the source somewhere on
the disc and then trigger the configuration stage. Here is how the sample
looks like:


  C:\XML\Win32> cscript configure.js prefix=C:\Home\Igor
    include=C:\Home\Igor\include lib=C:\Home\Igor\lib

  libxml2 version: 2.4.16
  Created Makefile.

  XML processor configuration
  ---------------------------
               Trio: Disabled
      Thread safety: Disabled
         FTP client: Enabled
        HTTP client: Enabled
     HTML processor: Enabled
    Catalog support: Enabled
    DocBook support: Enabled
      XPath support: Enabled
   XPointer support: Enabled
   XInclude support: Enabled
      ICONV support: Enabled
   Debugging module: Enabled
   Memory debugging: Disabled

  Win32 build configuration
  ---------------------------
      Debug symbols: Disabled
     Static xmllint: Disabled
     Install prefix: C:\Home\Igor
       Put tools in: $(PREFIX)\bin
     Put headers in: $(PREFIX)\include
   Put libraries in: $(PREFIX)\lib
       Include path: C:\Home\Igor\include
           Lib path: C:\Home\Igor\lib

  C:\XML\Win32>


Now you would type 'nmake' and libxml, xmllint and friends would be built.
Typing 'nmake install' thereafter would install the library. The configure
script accepts options that control which features are compiled in. Here:


  C:\XML\Win32> cscript configure.js help

  Usage:
    cscript configure.js <options>
    cscript configure.js help

  Options can be specified in the form <option>=<value>, where the value
  is either 'yes' or 'no'.

  Supported XML parser options, default value given in parentheses:
    trio:      Enable TRIO string manipulator (no)
    threads:   Enable thread safety (no)
    ftp:       Enable FTP client (yes)
    http:      Enable HTTP client (yes)
    html:      Enable HTML processor (yes)
    catalog:   Enable catalog support (yes)
    docb:      Enable DocBook support (yes)
    xpath:     Enable XPath support (yes)
    xptr:      Enable XPointer support (yes)
    xinclude:  Enable XInclude support (yes)
    iconv:     Enable ICONV support (yes)
    xml_debug: Enable XML debbugging module (yes)
    mem_debug: Enable memory debugger (no)

  Supported Win32 build options, default value given in parentheses:
    debug:     Build unoptimised debug executables (no)
    static:    Link xmllint statically to libxml2 (no)
    prefix:    Base directory for the installation (.)
    bindir:    Directory where xmllint and friends should be installed
               ($(PREFIX)\bin)
    incdir:    Directory where headers should be installed
               ($(PREFIX)\include)
    libdir:    Directory where libraries should be installed
               ($(PREFIX)\lib)
    include:   Additional search path for the compiler, particularily
               where iconv headers can be found (.)
    lib:       Additional search path for the linker, particularily
               where iconv library can be found (.)

  C:\XML\Win32>


The configuration script reads the version information from configure.in,
takes the command-line options into account and generates xmlversion.h from
xmlversion.h.in. The file xmlwin32version.h is practically identical to
xmlversion.h, so the configuration script creates it by simply making a copy
of xmlversion.h. It then creates the Makefile from Makefile.msvc, if needed.

The whole thing brings two new files to the source, 'configure.js' and
'Makefile.msvc', both in the Win32 subdirectory. This is a separate process
and does not affect the Win32/dsp subdirectory and its function in any way.
The configuration script even does not need to be MSVC-specific, it could be
tweaked to configure a mingw build, for example, if someone with a mingw
compiler would come and fix the long forgotten Makefile.mingw :-)

Oh, and the whole thing also applies to libxslt.

The configuration script is written in JScript, a MS implementation of the
ECMA 262 language. It is being executed through Windows Scripting Host, a
script driver which seems to be a part of the operating system since Win2k.
NT4 boxes should have it through the Internet Explorer and if not, well, the
configuration can be done manually and the .dsp files are still there as
well.

For those whose coffee break is still not fully spended reading this, here
is some background about why I am about to drop Visual Studio:

Like it was always the case, I was updating my development environment to a
newer version, as soon the newer version was available :-) This resulted in
installing the Visual Studio 7 (.NET) together with its glorious compiler
suite. While this IDE is very good for .NET development, I encountered few
flaws when compiling regular C/C++ programs:
* It links to a self-supplied C-runtime, called msvcr70.dll, which is
unavailable unless you install the IDE. This means that software compiled
with the IDE cannot run on production systems, say servers. Of course, this
can be changed with few tweaks, and I did that.
* Doing that was not enough. Whenever a floating-point is casted to a long,
the cast results in a call to a _ftol2 function, which is unavailable in the
regular C-runtime. The regular C-runtime has _ftol function for this purpose
and I have no idea where that _ftol2 is hacked in. Perhaps a piece of code
which somehow gets statically linked, who knows? I didn't investigate
further.
* Being a .NET application itself, the IDE's integrated debugger has no
chance to do any kernel-mode debugging, a thing I like and need to do right
now.
* The project files of the new IDE are not compatible with the old. One must
convert the old ones to the new format and decide which IDE to use for which
project. This implies, you must have both IDEs, 6 and 7 installed, no matter
what.

Well, since I need some of the features offered in the new compiler, I was
looking for a workaround. Since my updated machine does not have the old IDE
installed anymore, I ended up writing a makefile for libxml and other stuff
which is programmed in plain C/C++. Now, I was about to install the old IDE
back, but unfortunately I figured out that my productivity increased during
my time without it. I suddenly found myself writing scripts to automate
tasks which I dickheadly did manually again and again for months. They say
that xemacs cannot make you coffee, but that's about all it cannot do.
Indeed, it seems that making coffee is all that's left for me to do myself
now :-)

In any case, I think it better to have build files whose format does not
change very often. Just as a comparison, it is true that one needs to read
an 800-page manual to be able to remotely use emacs. It is also true that
one needs to do it only once and it lasts for life, for while emacs gained
functionality, it didn't significantly change its user interface during all
these years of its existence. Visual Studio changes drastically with every
version, in every aspect, and while you need a smaller manual, you need a
new one every now and then. Don't take me wrong, I have nothing against
change, I am changing something every week. But I do have a lot against the
need to maintain everything several versions back, because there are always
people out there using an older version. Well, makefile format and C
compiler command-line syntax is less prone to radical changes than the IDE
and instead of maintaining several totally different versions, one can
maintain one compatible version. After all, makefile is understandable and
readable for everyone, no matter which platform she uses.

Still here? Thanx for listening :-) What I would propose now is to adapt
this new build process into libxml and hopefully make it the mainstream. The
files in the win32/dsp subdirectory should of course remain there, for those
who need or want them. I will still be there to help fixing them if
necessary, but I will probably hear about something being wrong when someone
else reports it, because it seems I won't be using Visual Studio IDE for
C/C++ projects.

I would like to hear comments, specifically from NT 4 users about the
availability of the JScript engine. If everything comes positive, I will
post the files here. I would just like to check the build with the newest
source first and that I cannot do because CVS seems to be down at the
moment.

Ciao
Igor