Status of build infrastructure transition

[Daniel Elstner <daniel kitta googlemail com>]

Hi all,

as announced, the C++ binding modules are currently transitioning to an
improved build infrastructure.  This involves a lot of grunt work, but
when it's over we will finally have the documentation building correctly
for everyone out of the box, with correct references to external
documentation and customizable translation to local paths at install
time.  It will also be easier to set up the build for a new C++ binding
module.

The heavy groundwork is now finished and we have a foundation to build
upon for the transition of the remaining binding modules.  So far, a new
module mm-common has been created, and the transition of both libsigc++
and glibmm is essentially complete.  I'm now going to explain the new
stuff in detail.

mm-common
=========

A new module "mm-common" has been created on git.gnome.org, similar in
spirit to the "gnome-common" module.  It will be a hard dependency when
building from git for every module converted to the new system.  It will
not be required building from release tarballs, unless maintainer mode
is enabled explicitly.  It is not pretty, and we have resisted creating
something similar to gnome-common for a long time.  However, compared to
the massive amount of copy'n'paste between the C++ binding modules and
the difficulties caused by it, the mm-common solution turned out to be
the lesser evil.

1. libstdc++.tag

At 'make' time, mm-common downloads the latest libstdc++.tag file from
gcc.gnu.org using either curl or wget, whatever is available.  If you
have the choice, install curl as it supports Transfer-Encoding: gzip
natively which reduces the download size to about 12 % of the expanded
file.  The gcc.gnu.org web server provides time stamps, so the file is
only downloaded again if it has changed on the server.  The installed
location of libstdc++.tag is made available through pkg-config, so that
it can be shared by all C++ binding modules.

2. mm-common-prepare

Not all files shipped with mm-common can be package dependencies,
because some are required even for the tarball build, or should be
shipped with every module for other reasons.  This means that there now
is yet another little script to invoke from your autogen.sh, namely
mm-common-prepare, with the usual command line interface.

Running mm-common-prepare will install build support files into
AC_CONFIG_AUX_DIR() if specified in configure.ac, otherwise to the
top-level source directory.  It also looks for an argument to the custom
MM_CONFIG_DOCTOOL_DIR() macro, and copies the utility scripts needed for
the documentation build into the specified directory.

3. Automake include files

The mm-common module contains the Automake include files for generating
and building bindings and creating reference documentation with Doxygen.
These are meant to be shipped with every package that uses them and are
therefore copied into the source tree by mm-common-prepare.  Automake
will automatically take care of distributing any file that is included
from one of your Makefile.am files.

The interface is built around the same ideas as the old .am_fragment
files from the build_shared/ directory.  However, some restructuring
made them easier to use as well as more flexible if customization is
needed.  They are already somewhat better documented than the old
include files, but usage instructions are still missing.  However, if
you already knew the old include interface, you will be fine.  All
user-overridable variables and expected input variables are listed at
the top of each .am file, and most variable assignments are accompanied
by a short comment to explain their purpose.

4. Autoconf M4 macros

Also included is a collection of Autoconf M4 macros for configure-time
tasks specific to the *mm bindings.  In particular, the file mm-doc.m4
defines are number of macros which are essential for the operation of
the doc-reference.am include file.  All macro names start with the MM_
namespace prefix, or _MM_ for internal macros.  All macro prototypes are
documented.  Additionally, the interface of every public macro is fully
documented with usage instructions.

These macro files are always installed into ${datadir}/aclocal to be
picked up automatically, so there is no need for mm-common-prepare to
install the .m4 files into the source tree.  Nonetheless, each file has
a #serial number in YYYYMMDD format (base 10), so you could override
them within your source package with newer versions if you wanted to.

The most important macros for bindings are:

* MM_INIT_MODULE, which replaces the pushdef mess in every configure.ac
  by a one-liner.

* MM_ARG_ENABLE_DOCUMENTATION to provide the --enable-documentation
  configure option and check for required utilities.

* MM_ARG_WITH_TAGFILE_DOC, which implements the magic that ties all the
  Doxygen tag files together, prepares for translation into local paths
  on installation, and also provides a means for distribution packagers
  to fine-tune the locations pointed to.

For detailed documentation and the other utility macros which are
provided, see the .m4 files in the macros/ directory of mm-common.

5. doctool scripts

These scripts are required utilities for building the Doxygen reference
documentation using the new doc-reference.am include file.  These files
are copied into your package by mm-common-prepare only if you defined a
destination directory using the MM_CONFIG_DOCTOOL_DIR macro.  The glibmm
package installs these files into a location retrievable via pkg-config,
so that any binding module which already depends on glibmm does not need
to ship them and will use the scripts provided by glibmm automatically
unless explicitly overridden with MM_CONFIG_DOCTOOL_DIR.

These utility files are:

* doc-install.pl, which is a saner and more flexible replacement for the
installdox utility.  It can be used as a drop-in replacement for the BSD
install command executed by Automake.
* doc-postprocess.pl, which was renamed from beautify_docs.pl and has
seen only minor changes.
* tagfile-to-devhelp2.xsl, to translate a project's Doxygen tag file
into a Devhelp book, format version 2.  The XML output of Doxygen does
not have to be enabled anymore to build the Devhelp file, which speeds
up the build noticeably.  Also note that doc-install.pl has the ability
to translate the base path of a Devhelp book at install time.

All these utilities are executed automatically if doc-reference.am is
used.  The Devhelp book will always be generated and there is no need to
declare anything to get it.  For detailed documentation, see the files
build/doc-reference.am and macros/mm-doc.m4 in the mm-common module.

-----------

I probably forgot one thing or another, but the mail is already too long
as it is.  I should probably compile this text into a README document.

Tomorrow, David and I will begin with the conversion of binding modules
higher up in the stack, starting with cairomm and moving to gtkmm.

Cheers,
--Daniel