Build tools standard redux

[Malcolm Tredinnick <malcolm commsecure com au>]

The thread from a week ago that Dan Mills started about build tools
seems to have petered out without any clear resolution (in GNOME? How
surprising .. not). It cannot be left like that, since we are already in
trouble and need to work out a solution.

In this mail I have tried to summarise what different people's comments
were and also the current state of the tree for building things. It's a
bit long, because I am trying to pull everything together. I do not
really know what to do with this information -- it's not really ready
for a GEP, since there is no clear consensus (and the GEP process seems
to have tragically stalled in any case). It may ultimately need to be
decided by the release team, since it's part of the "keep CVS buildable
so that people can run it always" mission statement. However, before
that can happen, we need to be slightly less all over the shop.

Since I rarely have an original thought (apparently), I have tried to
just list known, indisputable facts in this mail -- reporting what
others have said, etc -- without injecting too much of my own opinions.

Also, I have just stuck to the topic of Dan's original post here: the
automake, autoconf and libtool packages. We can have the gettext
argument some other time.

Currently, building GNOME 2.2 from CVS requires the following:
	- automake 1.4 and automake 1.6
	- autoconf 2.57 (? -- needs to be strictly later than 2.53, but
	  not sure which intermediate version is sufficient. Prior to
	  31 Jan 2003, autoconf 2.53 was sufficient.)
	- libtool 1.4.2

[For reference, the requirements for GNOME 2.0.2 from CVS were the same,
except that autoconf 2.53 was sufficient.]

The latest version of all these tools, for reference, are:
	- automake 1.7.2 (released 5 Dec 2002)
	- autoconf 2.57 (released 3 Dec 2002)
	- libtool 1.4.3 (released 23 Oct 2002)

A back of the envelope survey (and mixed metaphor) shows that it is not
uncommon to have automake 1.6.x (Apr - Jul 2002), autoconf 2.53 (Mar
2002) and libtool 1.4.2 (Sep 2001) installed from recent releases of
dsitributions. GNOME requires automake-1.4 to be at least the '-p6'
version (Jul 2002) to allow parallel installs with automake 1.6 to work.

To summarise the previous thread (you'll have to search the archives
yourself if you want proof. No URLs here. I just grabbed the items out
of my desktop-devel mailbox) ...

	- Dan Mills pointed out the need for a consistent set of build
	  of tools that was known and conformed to. He was not a fan of
	  allowing multiple toolset versions even if they could be
	  parallel installed. He felt it was not unreasonable to fix the
	  versions, since they were only really fixed for six months or
	  so, over the course of one release.

	- Rodney Dawes tried to get things rolling by suggesting we
	  standardise on automake 1.6, autconf 2.53 and libtool 1.4.3.

	- Colin Walters wanted anything but automake 1.4 and suggested
	  using automake 1.7 if we were going to change (on the grounds
	  that he could see no reason not go to the most recent
	  version).

	- Havoc Pennington claimed that once we had something like
	  gnome-hello building with automake 1.7, everything would be
	  fairly easy to move across (basically, there are issues with
	  the doc building stuff). [Editor's note: this claim was not
	  met with the cries of disbelief I expected, so it may even be
	  true. An earlier thread on this topic a few months ago was not
	  so charitable.]

	- James Henstridge wanted to use automake 1.7.2. He said he
	  wanted to able to use the latest tools in his modules and
	  specifically objected to using autoconf 2.53, since it was 11
	  months old and there were newer versions around and older
	  automakes for the same reason.

	- Jonathan Blandford put in a request to know if there were any
	  bugs in the tools we were using that _required_ us to upgrade
	  (as opposed to just future-proofing the process). A few
	  problems with automake 1.4 were pointed out. Nothing for
	  autoconf or libtool were mentioned.

	- Jonathan also noted that keeping the requirements to a minimum
	  (i.e. low version numbers) was preferable, since that
	  increased the chances that people already had the required
	  tools installed on their system. James felt this was not such
	  a requirement, since people used build scripts anyway and
	  installed local copies of the tools. Dan was not convinced by
	  this latter argument, since he felt it was using the existence
	  of the problem to perpetuate the continuance of the problem
	  (although he did not use such big words to say it).

	- Michael Meeks did not like any tool that required the build
	  process to work with a write-protected source directory, since
	  that made for trouble with things like glib-genmarshall,
	  gtk-doc and the ORBit2 pre-processor. [Ed. note: automake 1.6
	  and later run 'make distcheck' with srcdir being write
	  protected.]

Although the above opinions were the only concrete ones expressed in the
thread Dan kicked off, they are obviously not the only ones out there. A
lot of people have opinions, but many of them will just be agreements
with one or more of the above and posting "me too" to the list is not
standard practice. Also, this topic is revisited every six to eight
months on the mailing lists, so much of it has been rehashed previously
(with, again, no really satisfying conclusions).

Some obstacles still exist to settling on a common toolchain:

	(1) The autoconf and libtool releases are not parallel
	installable. That is, you need to pick _a_ version of these and
	use it. On the other hand, it is possible to have multiple
	automake versions installed, but you then need to run through
	binary names checking to see which of automake-1.7, automake-1.6
	and automake-1.4 is available.

	(2) Some packages will need to branch for previous GNOME
	releases (e.g. 2.2) so that changes on the trunk do not
	adversely affect previous releases. Somewhere between the 2.0.0
	and 2.0.2 release of GNOME, we added a hard dependency on
	automake-1.6 for building from CVS because some modules had not
	branched.

	(3) As noted above, there is not clear consensus. it appears to
	come down to one of two cases:
		(a) Keep using tools that are distributed by vendors to
		ease the pain on people building from CVS.

		(b) Fix on recent version of each tool and then do one
		big pass through the tree fixing up every single module
		in the platform release so that it works with the
		required tools. This is similar to the work done by
		Havoc when allowing automake-1.4 and automake-1.6 to be
		used in parallel, except that it's a bigger job.

Personal opinion time...

We MUST come up with a decision here. It would be a terrible mistake to
have the proposed action being "*shrug*". It is not an option to say
"too hard"; there are a number of not intolerable resolutions available
and the hard part is selecting one and sticking to it.

In view of point (1), above, and the fact that module maintainers can
currently choose whatever tools they like, the situation is currently
untenable. it will not be long before a maintainer who wants recent
tools and a maintainer who wants to maintain compatibility with older
tools become so far apart that building both packages on one
installation is impossible. Presently, even the most well-intentioned
maintainer can be left wondering what to do here, since there are no
guidelines, let alone requirements.

I am not in a position to drive the discussion much more than I have
done here because I do not maintain any significant modules.

I was prompted to write this because I did a complete rebuild for the
first time in a while and needed to update my toolchain. I consider
myself to be very comfortable with auto*, but I consider it an
unnecessary annoyance to have to debug problems of this nature. So
people for whom these tools are a bit frightening (and that is not a
small crowd) are going to be really put out.

Malcolm

-- 
Borrow from a pessimist - they don't expect it back.