Generic documentation build infrastructure

[Daniel Elstner <danielk openismus com>]

Hi all,

during the past couple of weeks I have been working on a new generic
build infrastructure for the C++ binding modules.  This work grew out of
my attempt to build Doxygen reference documentation for librsvgmm.  The
ad-hoc nature of the current "process" became clear pretty quickly, and
I began to work on a proper infrastructure intended to be shared by all
*mm modules to address the problem in a generic manner.

So, here is what we have right now:

* No generic infrastructure at all for Doxygen integration. The build
rules in docs/Makefile.am are usually copied as is into new modules and
then hand-edited.

* It is a bloody mess. Right now, without manual intervention, linking
to external documentation and generating the public web documentation
only works Murray. For everything to be generated properly, you need to
have the right versions of dependent modules checked into implicitly
assumed locations on your file system, and also fully built with
documentation enabled.

* There is no proper mechanism whatsoever to deal with references to
external documentation, in particular Doxygen tag files.  The generated
tag files are not installed at all with most modules.  The few modules
that do install their tag files use random file names and a random
installation location. No means are provided to find out about these
configuration details through pkg-config or another mechanism.

* Despite all these shortcomings, the stuff that is there (basically
only the cut'n'pasted docs/Makefile.am and Doxyfile.in) is by no means
easy to use by module maintainers. It's messy, and it doesn't really
work either.

I'm sure I could think of some more points, but that's enough for
now. :-)

Well, as I soon found out, there is a reason for all this mess: Handling
all of documentation generation, installing the documentation on the end
user's file system, providing the necessary features required by package
maintainers for distribution integration, and making the documentation
suitable for uploading to the public web server is a hard problem.  It
took several iterations to come up with something that is sufficiently
generic, reasonably easy to use by maintainers, flexible enough for
package maintainers and casual installers alike, and yet not overly
complicated in its implementation.

I realize that this comes up basically out of the blue for many people,
and that I haven't discussed it with anyone until now.  I'm not going to
touch a module without agreement by its maintainer.  But realistically,
the new infrastructure should be adopted by as many C++ binding modules
as possible in one go, in order to reduce the pain of the inevitable
transition period.

If we get the thumbs up from module maintainers, David King and I are
going to get this all over with *this week*.  You'll have to entrust us
with your baby for a short while.  Then we will see whether it worked
out as intended, and respond to the questions and suggestions from the
maintainers we forced our luck upon.

Implementation overview
=======================

* build/compile-binding.am:
* build/generate-binding.am:
These are Automake include files and replace the functionality of the
*.am_fragment files in the build_shared/ directory. This is not part of
the documentation infrastructure and therefore optional.

* build/dist-changelog.am:
Optional Automake include file with a rule to generate the ChangeLog
file from the git log at make dist time.

* build/dk-warn.m4:
The DK_ARG_ENABLE_WARNINGS() macro, which is already used by a number of
modules. Optional, of course, if you do not already use it anyway.

* build/doc-install.pl:
Now it gets to the meat. This file is a full-blown replacement of the
installdox script generated by Doxygen.  Just as installdox does, it
translates references to external documentation at installation time.
This is necessary because the documentation is included in the tarballs,
but at the time of generation the location of external documentation is
not known.
Apart from replacing the functionality of installdox, doc-install.pl
also acts as a drop-in replacement for the classic 'install' command
used by Automake rules to install files. It is possible to tell Automake
to use doc-install.pl instead for a subset of files, which avoids the
need for custom installation rules in the Makefile.am.
It can translate Devhelp books as well, and will happily pass through
unrecognized files.

* build/doc-reference.am:
Automake include file for building the Doxygen reference documentation.
In the same spirit as the other Automake include files, this is intended
to implement the bulk of the work in a generic manner with only a few
variables to be adjusted from the including Makefile.am.

* build/mm-doc.m4:
The MM_ARG_ENABLE_DOCUMENTATION Autoconf macro that relieves you from
cut'n'pasting broken bits of shell code from random configure.ac files
in the wild.  It takes care of providing the --enable-documentation
option, checking whether the necessary tools are installed, providing an
Automake conditional, etc.
The other Autoconf macro defined here is MM_ARG_WITH_TAGFILE_DOC.  This
is sort of the heart of the new infrastructure for referencing external
documentation, and ties all the ends together.  You call this macro once
for each external Doxygen tag file your module depends on.  It provides
a configure option to override tag file locations, attempts automatic
configuration if possible, and takes care of building the list of tag
files for Doxyfile, as well as the command line arguments to installdox
or doc-install.pl.

* build/mm-module.m4:
The magic MM_INIT_MODULE macro intended to relieve us from the
repetitive pushdef() mess I have noticed in many binding modules.

* build/mm-pkg.m4:
The helper macro MM_PKG_CONFIG_SUBST, which simplifies the retrieval of
specific configuration values from pkg-config.

* doc/reference/doxytag_to_devhelp2.xsl:
Brand-new and shiny rewrite of the XSLT script used to generate the
Devhelp book for the reference documentation.  It outputs Devhelp2
format now, and no longer requires the Doxygen XML output at all.
Instead, all the information that is needed is read from the tag file
generated by Doxygen for the HTML documentation.

To see how it all fits together, check out the librsvgmm master branch
on git.gnome.org: <http://git.gnome.org/cgit/librsvgmm/tree/>

Phew.  Sorry everyone for boring you to death.  Now it is up to module
maintainers to give us the go-ahead for breaking everything.  (The idea
is to convert only the master branches for now, and then we will see.)

Digest!
--Daniel