[gnome-love] Building GNOME HOWTO

[Willie Walker <William Walker Sun COM>]

Hi All:

I went through great pains getting GNOME to compile.  It was kind
of a harrowing experience, but I've finally landed on Fedora Core 3
and had some decent success with it.

I took a lot of notes along the way and spent this morning writing
things up in TWiki style.  It may still be a little rough, but
hopefully this will be of use to someone.  My vision for it is
as follows:  I did the preamble and such and wrote up my experiences
for Fedora Core 3; others might want to add their own notes for
their specific experiences on their distributions.  I also envision
creating a GNOME Developing HOWTO that walks people through the
pain of now actually working with and modifying a fresh build.

In any case, hopefully this can be of use to someone,

Will

---+ Building GNOME HOWTO

This is the BuildingGNOMEHowTo page.

%TOC%

This page is intended to assist you in building GNOME entirely from
the sources in the
[[http://developer.gnome.org/tools/cvs.html][GNOME CVS Repository]].
It is meant to help "first time" GNOME developers get
through the frustrating learning curve of knowing what to use and how
to use it.



---++ Why Build From Sources?

If you are going to be contributing to a module or you plan on being a
maintainer of a module, you generally should try to work with the
latest sources to ensure that your stuff builds and works with the
latest code.



---++ GNOME CVS Repository Meta Structure

The [[http://developer.gnome.org/tools/cvs.html][GNOME CVS Repository]]
page contains a lot of good information about the structure of the
GNOME CVS Repository, along with a description of some of the common
conventions that various module developers use.  Even if you are
already familiar with CVS, the GNOME CVS Repository page is worth  
reading.

GNOME is also broken into many releases which are managed via tags and
branches in the CVS repository.
The [[http://www.gnome.org/start/][Getting Started with GNOME]] page
provides a description of the current releases and future plans.



---++ Setting Up Your Build Environment

Although your build environment can be anything you want, this
document describes one way that has been shown to work well.  For the
purposes of this HOWTO, we'll examine the user _wwalker_ who is building
GNOME under his home directory.

When building GNOME, you will want two main directories: one to hold
the sources and build output, and another to serve as the
"installation."

The "installation" represents the actual directory structure that one
might see had they installed a released package.  For example, it
contains the applications, the shared libraries, documentation, etc.

Some insane users will attempt to install the build output directly
under /usr, replacing the packages that came with their Linux
distribution.  If you want to play with fire, you can do this, too.
If you want to be on the safe side, then set aside a separate
directory for your installation.

The user wwalker has created the following directories:

   * ~/work/gnome - holds the sources and build output
   * ~/work/gnome-install - holds the "installation" output

A complete build of gnome requires about 2.75G for the source/build
directory and about 600M for the installation directory, so please
plan accordingly.

NOTE: at this point, wwalker has only created the directories -- he
hasn't downloaded anything from the GNOME CVS repository yet.



---+++ Setting up JHBuild

As you can tell from browsing the
[[http://cvs.gnome.org/viewcvs/][online CVS repository]], GNOME
consists of many modules.  Downloading and building each of these by
hand can be quite cumbersome.  In addition, many of the modules have
dependencies on each other as well as other "external" (i.e., not in
the GNOME CVS repository) components.  Without knowledge of what to
install, and the order in which to build things, you can easily go
crazy attempting to build things "by hand."

Luckily there is a tool,
[[http://www.gnome.org/~jamesh/jhbuild.html][JHBuild]], that
understands the various dependencies of all the modules.  In addition,
this tool is smart enough to be able to automatically download, build,
and install all the modules in order and with the right flags.

Get to know and love JHBuild.  While it may feel like your enemy at
first, it can actually become a very good friend over time.

To set up JHBuild, you first need to download it from the GNOME CVS
repository.  Doing so requires you to "login" to the repository
as an anonymous user (when prompted for a password, do not type
anything, just press return), and to then check out the jhbuild module:

<verbatim>
# cd ~/work/gnome
# cvs -z3 -d:pserver:anonymous anoncvs gnome org:/cvs/gnome login
# cvs -z3 -d:pserver:anonymous anoncvs gnome org:/cvs/gnome co jhbuild
</verbatim>

You will see a bunch of output like the following:

</verbatim>
cvs server: Updating jhbuild
U jhbuild/.cvsignore
U jhbuild/COPYING
...(stuff deleted for brevity)...
U jhbuild/scripts/gnome-2.6.deps
U jhbuild/scripts/gnome-2.6.exceptions
U jhbuild/scripts/mk-tarball-moduleset.py
</verbatim>

At this point, all you need to do is build and install JHBuild:

<verbatim>
# make
gcc -Wall -O2 -o install-check install-check.c
Run "make install" to install.
# make install
Creating /home/wwalker/bin/jhbuild
Creating /home/wwalker/.gnome2/vfolders/applications/jhbuild.desktop
Don't forget to create ~/.jhbuildrc
install -m755 install-check /home/wwalker/bin/install-check
</verbatim>

This places a "jhbuild" binary in your ~/bin directory.  So, for
convenience, you should add ~/bin to your PATH.  In addition, you need
to make a ~/.jhbuildrc file.  This tells JHBuild very important
information, such as what version of GNOME you want to build, where to
put the sources and build files, and where to put the installation.
Here's an ExampleJHBuildRC (WDW: make this a Wiki link and move
the ~/.jhbuildrc contents there) that will build the latest sources
(Gnome-2.10) from the GNOME CVS Repository:

<verbatim>
# -*- mode: python -*-

# edit this file to match your settings and copy it to ~/.jhbuildrc

# if you have a cvs account, uncomment this line
#cvsroots['gnome.org'] = ':pserver:user cvs gnome org:/cvs/gnome'

# what module set should be used.  The default at the moment is  
'gnome28',
# but it can be any of the files in the modulesets directory, or even
# the URL of a module set file on a web server.
moduleset = 'gnome-2.10'
#moduleset = 'gnome28'

# A list of the modules to build.  Defaults to the Gnome Desktop and
# developer platform.
modules = [ 'meta-gnome-desktop', 'meta-gnome-accessibility' ]

# what directory should the source be checked out to?
checkoutroot = os.path.join(os.environ['HOME'], 'work', 'gnome')

# the prefix to configure/install modules to (must have write access)
prefix = os.path.join(os.environ['HOME'], 'work', 'gnome-install')

# extra arguments to pass to all autogen.sh scripts
# to speed up builds of gnome2, try '--disable-static --disable-gtk-doc'
autogenargs='--disable-static'

# set CFLAGS:
#os.environ['CFLAGS'] = '-g -O2'

# a alternative install program to use.
# The included install-check program won't update timestamps if the
# header hasn't changed
os.environ['INSTALL'] = os.path.join(os.environ['HOME'],
                                     'bin', 'install-check')

skip = ['mozilla']
tinderbox_outputdir = os.path.join(prefix, 'log')
</verbatim>

There are a few important things to note here:

   * moduleset - this tells JHBuild what to pull from the GNOME CVS
     Repository.  That is, which modules it should care about and
     which revisions of the files it should get.  It also defines the
     dependencies between the modules so you don't have to worry about
     it.  If you look in the ~/work/gnome/jhbuild/modulesets
     directory, you'll see several modulesets.  The naming scheme is a
     little inconsistent, but you'll see one for the GNOME 2.4 release
     (gnome24.modules), the GNOME 2.6 release (gnome26.modules), the
     GNOME 2.8 release (gnome28.modules) and the upcoming GNOME 2.10
     release (gnome-2.10.modules).  The moduleset you specify is
     merely the name of one of these files with the ".modules"
     extension dropped.  Since he wants to work with the latest
     sources, wwalker has chosen the "gnome-2.10" moduleset.

   * modules - this tells which subset of the moduleset JHBuild should
     build.  If you dig into the moduleset file you chose, you'll be
     able to see the list of all the modules.  The
     [[http://www.gnome.org/~jamesh/jhbuild.html][JHBuild Manual]]
     provides a good description of the moduleset file format.
     Since he plans on contributing to the GNOME accessibility
     work, wwalker has chosen to build the "meta-gnome-desktop" and
     "meta-gnome-accessibility" modules.

   * checkoutroot - this tells JHBuild where to put the sources it
     gets and where to do the build.  wwalker has chosen ~/work/gnome.

   * prefix - this tells JHBuild where to install the modules is has
     built.  wwalker has chosen ~/work/gnome-install.

   * autogenargs - this tells JHBuild what to pass to autogen.  You
     need not really know what autogen does, but the
     "--disable_static" flag comes highly recommended as an option to
     speed up the build.

   * skip - this tells JHBuild to specifically skip building some
     modules.  Since it takes a long time to build, and since wwalker
     doesn't plan on working with it, wwalker has chosen to skip the
     "mozilla" module.

   * tinderbox_outputdir - tinderbox is a feature of JHBuild that
     produces HTML output of the build results.  The
     tinderbox_outputdir tells JHBuild where to put the results of
     tinderbox.  wwalker has chosen ~/work/gnome-install/log, but you
     can choose anything you want.  Make sure you actually create this
     directory - JHBuild will not make it for you.

At this point, you are almost ready to run, but you still have a few
more things to set up with JHBuild.  What you need to do varies by
your Linux distribution, so these have been broken into the following
separate sections based on the type of Linux distribution.



---++ Fedora Core 3

These instructions were written based upon an installation of Fedora
Core 3 where *all* possible packages from the distribution have been
installed.

NOTE: Allow for about 5 hours to build GNOME from scratch.  Also, you
need a BIG sense of humor.  Most module maintainers try to be quite
good about not checking things in that are broken, but things still
have a tendency to break, especially if you are using the latest
moduleset.  Furthermore, what built today may not always build
tomorrow.  So...don't go insane.  When something breaks, turn to
others for help (see the bottom of this page for the IRC chat room
information).



---+++ Finish JHBuild Configuration

At this point, all that has been assumed is that you have gone through
the initial setup of JHBuild.  Now, you need to do more work with
JHBuild because it requires more tools to work.  This is accomplished
via "bootstrapping".  To do this, execute the following command from
any directory (assuming ~/bin is in your PATH)

<verbatim>
# jhbuild bootstrap
</verbatim>

This will take about 15 minutes to run, and will download, build, and
install a bunch of components needed by JHBuild and the build
process.  Everything gets placed into the "checkoutdir" and "prefix"
directories you specified in ~/.jhbuildrc.

At this point, you should be good to go, but just to be safe, try the
following command:

<verbatim>
# jhbuild sanitycheck
</verbatim>

It should output nothing.  If it outputs something, then it's telling
you what's missing.  On Fedora Core 3, sanitycheck should work just
fine after a bootstrap has been done,



---+++ Build GNOME

You're now ready to build GNOME for the first time!  Starting the
build process is very simple:

<verbatim>
# jhbuild tinderbox
</verbatim>

This starts the build process, and JHBuild will automatically pull
modules off the network (including GNOME CVS and other places), build
them, and install them.

JHBuild's tinderbox option will create an index.html and other *.html
files in the directory you specified as your tinderbox_outputdir in
~/.jhbuildrc.  Remember that you have to create this directory by
hand; JHBuild will not create it for you.

To monitor the build progress, point your browser to the index.html
file.  JHBuild will update index.html each step of the way.  Green
means good.  Red means bad.  If you want to get an idea of just how
far along the build is, you can run the following command to get a
complete list of all the modules that will be built:

<verbatim>
# jhbuild list
</verbatim>

NOTE: You might get the following warning in the build logs for some
of the modules:

<verbatim>
   ...alocal/pkg.m4:5 warning: underquoted definition of  
PKG_CHECK_MODULES
</verbatim>

It's OK to ignore this warning.



---++++ Fix Howl

Howl has been introduced in 2.10.  It will break the build because it
depends upon static libraries.  You'll notice its failure by seeing
nice red spots in the lovely HTML produced by tinderbox.  Since howl
is so deeply nested in the dependency list, it's failure to build will
also cause most of the rest of the build to fail.

So...keep an eye on your tinderbox.  When you see howl fail, kill
your "jhbuild tinderbox" process ("Ctrl-C" should work, but you might
need to use "Ctrl-Z" followed by "kill -9 %1" to kill it).  Then
perform the following steps:

<verbatim>
# cd ~/work/gnome
# rm -rf howl-0.9.8
# gunzip -c howl-0.9.8.tar.gz | tar xvf -
# cd howl-0.9.8
# jhbuild shell
# ./configure --prefix=/home/wwalker/work/gnome-bld
# make
# make install
# jhbuild tinderbox --start-at=howl
</verbatim>

Alternatively, you might try the instructions at
[[http://bugzilla.gnome.org/show_bug.cgi?id=161153][Bugzilla Bug  
161153]]
(the author of this page found that these instructions did not work).



---+++ Congratulations

If you made it this far, you should have a build that works.  Phew.
You probably want to keep things kind of safe - you can do this by
preventing JHBuild from grabbing the latest sources from the network
by adding the following line to ~/.jhbuildrc:

<verbatim>
nonetwork = True
</verbatim>



---++ CCache

[[http://ccache.samba.org][CCache]] is an interesting tool that is
meant to help speed up your subsequent builds of GNOME.  People have
had good results with it.

To build and install CCache:

<verbatim>
# gunzip -c ccache-2.4.tar.gz | tar xvf -
# cd ccache-2.4
# ./configure
# make
# make install (creates /usr/local/bin/ccache; you'll need to be root)
</verbatim>

Now, you'll need to set up CCache for your personal use.

<verbatim>
# cd ~/bin
# for cmd in cc gcc c++ g++; do
# ln -s /usr/local/bin/ccache $cmd
# done
</verbatim>

After installing CCache, set the cache size with the following
command.  wwalker chose 4G since he has a lot of free disk space.

<verbatim>
# ccache -M 4G
</verbatim>

You can check the status of your cache (such as cache hit rates) with
the following command:

<verbatim>
# ccache -s
</verbatim>



---++ More Help and Documentation

Hopefully, things will go smoothly for you.  If they don't, there are
other places to turn.  As you work, please take really detailed notes
and contribute them back to the community.  Over time, we can help
each other to faster success by spreading the knowledge.



---+++ IRC Chats

You can use GAIM to get to "irc.gnome.org," which has several channels
where very helpful people often hang out.  The two channels that might
be most useful are as follows:

   * #gnome-love - very good at helping first timers work through issues
   * #gnome-hackers - for the more experienced people

If you are new to IRC, you might want to read the
[[http://www.newircusers.com/irccmds.html][New IRC Users]] page.



---+++ GNOME Docs

   *  
[[http://www.gnome.org/~newren/tutorials/developing-with- 
gnome/][Developing in GNOME overview]] -
     people like to you point you here.  It helps some if you want to
     develop new GNOME apps.

   * [[JHBuild Manual][http://www.gnome.org/~jamesh/jhbuild.html]] -
     print this out, read it, write on it, highlight the things
     important to you.



---+++ Running With Your New Build

Well...it's most likely that you didn't go through this whole process
just to build GNOME and be done with it.  You most likely want to edit
some sources and test your changes using the stuff you just built!
See the DevelopingGNOMEHowTo page for more information (WDW - to be
written).