Building glib-2.0.4 with mingw and autotools on cygwin (long posting)

[Andreas Ames <andreas ames Tenovis com>]

Hi,

I'm using glib within an own application (a jabber client lib) because
of data types and portability support.  My first target is windows, so
I wanted to manage the build process on this platform (for example to
be able to produce debug symbols for glib).  Below I give a recipe how
to build glib-2.0.4 on cygwin using standalone mingw and
'autoconfiscation' because I didn't find one but read in the archives
that such descriptions are also needed by other people.  BTW, thanks
for this good piece of software.

First thing I tried, was to use msvc makefiles.  After some fiddeling
the make process ran through and I had a usable dll.  Unfortunately
there was a nasty bug:  a file handle opened with 'open' wasn't
accepted by 'fstat' (both in the microsoft runtime).  Although I
haven't further investigated I think the reason is that the dll linked
in both msvcrt.lib and msvcrtd.lib.  'open' was resolved from
msvcrt.lib and 'fstat' was resolved from msvcrtd.lib.  I believe this
is because the makefile.msvc in the build/win32/dirent subfolder
doesn't honor debug settings in modules.def/make.msvc and links always
against msvcrt.lib.  So take care, if you want to build with msvc.

Although this could be easily fixed there is no generic way to build
with msvc (when upgrading the glib source tree, you'll always have to
perform manual intervention).  So I decided to try a mingw build with
autotools.  That resulted in me fighting libtool for almost a week
which is not only libtool's fault but also mine because till now I had
no idea how to use the autotools as a developer ;-).

1) System setup

I have reinstalled a very late cygwin into the root directory of an
own partition (i.e. I didn't use the usual 'c:/cygwin' but just 'f:/'
as installation folder).  This makes things easier when using cygwin
tools like make, bash etc. together with native compilers like mingw
or cl because all paths on this partition are understood by both
cygwin and non-cygwin tools.  Note that the cygwin guys explicitley
discourage this way of installation.  Till now I had no problems with
this setup and I hope that most of the issues will be circumvented by
not using 'c:/'.  You can probably manage the build process with
cygwin in the default 'c:/cygwin' but you will have to perfom some
extra steps.  If you choose to try, here are two hints:

* When you provide additional include (-I) and library (-L) paths to
  the configure script, specify every path twice:  once in cygwin
  syntax and once again in windows syntax.

  Example:

  CPPFLAGS='-I/usr/local/include -Ic:/cygwin/usr/local/include' \
  ./configure

  (Note that this is *one* commandline.)

* You will almost certainly have to patch the libtool script,
  esp. where it uses 'pwd' to determine the path of some file that is
  further given to one of the mingw tools as a command line argument.

  Example:

  There are some places in libtool that look like
  'xabs=`pwd`"/$xlib"'.  Some lines later on, 'xabs' is used as an
  argument to 'ar' like in '$run eval "(cd \$xdir && $AR x \$xabs)"'.
  Immediately before this call to 'ar' you will have to do something
  like 'xabs=`cygpath -w \$xabs`'.


2) Versions

I use following versions of the tools:

* mingw 1.1 installed to f:/mingw

* the autoconf-devel package of cywin (2.53a)

* the automake-devel package of cygwin (1.6.2)

* the libtool-devel-20020705 package of cygwin


3) Dependencies

Till now I didn't manage to build libintl by myself.  So I use the
binary package as provided by the gimp-win32 guys.  Unfortunately this
dll also has been linked against both msvcrt.lib and msvcrtd.lib.  I
will try to get it built with mingw within the next days.

I use libiconv-1.8 built with msvc.  Although this one seems to be
clean I will try to rebuild it with mingw.

Unfortunately the autoconf step of glib produces an unfortune
'sys_lib_search_path_spec' in the libtool script (I don't think this
is glib's fault, I think autoconf is broken here).  The problem is
that cygwin library directories are included esp. '/lib'.
Unfortunately this one contains export libraries of both libintl and
libiconv.  Normally this should be no problem because the paths
specified on the commandline are searched first.  Unfortunately it
seems that libtool searches all given paths for .la files before it
looks for .a files in all paths.  Cygwin provides such .la files for
its versions of both libintl and libiconv but neither the binary
package of libintl nor the msvc built libiconv contain such files so
that libtool 'thinks' it should use the cygwin ones.  This would make
the resulting glib GPLed implicitely and I wouldn't be allowed to use
it.  So I simply produced my own .la files.  I'm sure you could make
libtool generate them for you, but I don't know how.  Therefore I
wrote them with an editor.  Both of them are attached.  If you use
them you need to need to adjust the 'libdir' entry.  When I manage to
build libintl and libiconv autoconfiscated with mingw this step won't
be necessary anymore.

An alternative to the .la files could be to correct the value of
'sys_lib_search_path_spec' in the libtool script but I didn't try that
one; it might be easier than creating the .la files.


4) Get new autotools

These are the needed commands (execute them in cygwin bash):

$ cd glib
$ tar-jxvf glib-2.0.4.tar.bz2
$ cd glib-2.0.4
$ mv configure configure.old
$ aclocal
$ autoconf -f
$ automake -a -c -f
$ libtoolize -c -f


5) Configure the package

These are the needed commands:

$ mkdir ../build_debug
$ cd ../build_debug
$ export PATH=/mingw/bin:$PATH
$ CC='gcc -mpentium -fnative-struct' \
CPPFLAGS='-I/home/ama2fr/projects/webti_events/libiconv-1.8/include \
-I/home/ama2fr/projects/webti_events/libintl/include' \
LDFLAGS='-L/home/ama2fr/projects/webti_events/libiconv-1.8/lib \
-L/home/ama2fr/projects/webti_events/libintl/lib' \
../glib-2.0.4/configure --with-libiconv --disable-static \
--prefix=/home/ama2fr/projects/webti_events/local \
--host=i386-pc-mingw32 --enable-maintainer-mode

Note that you have to put the last command completely into one line,
i.e. you can't just copy and paste it in one shot, but rather line by
line.  Anyway you will need to customize the paths.

The configure script warns about four things:

* configure: WARNING: If you wanted to set the --build type, don't use
--host.  If a cross compiler is detected then cross compile mode will
be used.

* configure: WARNING: I can't find the MACRO to enable thread safety
                on your platform (normally it's _REENTRANT). I'll not
                use any flag on compilation now, but then your
                programs might not work.  Please provide information
                on how it is done on your system.

* configure: WARNING: the
                'g_get_(user_name|real_name|home_dir|tmp_dir)'
                functions will not be MT-safe during their first call
                because there is no working 'getpwuid_r' on your
                system.

* configure: WARNING: the 'g_date_set_time' function will not be
                MT-safe because there is no 'localtime_r' on your
                system.

The first warning should be rather unimportant.  For the other three I
would like to hear (by the maintainers or other people) what they mean
for the thread-safeness of my application (esp. because the functions
in msvcrt.lib should be thread-safe).  Anyone?


6) Patches

At first patch the libtool script as follows:

$ patch <glib-2.0.4-libtool.patch

The patch is attached.  Please do yourself a favour and don't apply
this patch blindly against other versions of glib (for example the
2.0.6, which I didn't try out yet).  I think libtool is broken here
but the patch is certainly not general enough to propose it to the
libtool maintainer because I know far too few about libtool.  I only
know that this patch works for glib-2.0.4 (hopefully, that is :-).

There is another small thing left to be done.  In the following
Makefile's at the given line there is a call to msvc's lib.exe to
generate an export library.  This call is just 'lib' which doesn't
work in my bash.  This may be because I have it badly configured or
because my current version is buggy or because of something completely
different.  By just replacing the string 'lib' (only the first
occurency please!) with the string 'LIB.EXE' everything worked for me.

* gthread/Makefile:484

* gobject/Makefile:867

* gmodule/Makefile:522

* glib/Makefile:863

I didn't bother to write a script for that because it could possibly
be integrated into the correspondig Makefile.am's by the glib
maintainers, if it turns out that it is not just a local problem with
my setup.  It shouldn't break anything that worked till now anyway.

7) Workaround for Makefile shortcomings

Because I don't build within the source tree but in a separate build
tree (which makes it easy to concurrently maintain a debug and a
release build), I found some shortcomings in the Makefile's of glib,
which use special files that are located in the source tree but can't
be relocated to the build tree by the usual make mechanisms.
Therefore they are simply not found during the build.  You can perform
the following steps to cope with that (if your build tree and your
source tree are the same you can forget the following):

$ cp ../glib-2.0.4/glib/glib.def glib
$ cp ../glib-2.0.4/gobject/gobject.def gobject
$ cp ../glib-2.0.4/gmodule/gmodule.def gmodule
$ cp ../glib-2.0.4/gthread/gthread.def gthread

If you want to 'make install' there are still some other files to be
copied:

$ cp ../glib-2.0.4/glib/libcharset/ref-add.sed glib/libcharset
$ cp ../glib-2.0.4/glib/libcharset/ref-del.sed glib/libcharset

8) Make it

Finally you should now be able to make and install glib:

$ make
$ make install


There are some minor issues left which I couldn't solve till now.  One
is that mingw exports global variables regardless of the def-file.
Does anyone know of a way to get around this, i.e. no export for
global variables?


HTH,

andreas ames



# localcharset.lo - a libtool object file
# Generated by ltmain.sh - GNU libtool 1.4e (1.1125 2002/06/26 07:15:36)
#
# Please DO NOT delete this file!
# It is necessary for linking the library.

dlname='libintl-1.dll'
library_names='libintl.dll.a'
libdir=/home/ama2fr/projects/webti_events/libintl/lib

# localcharset.lo - a libtool object file
# Generated by ltmain.sh - GNU libtool 1.4e (1.1125 2002/06/26 07:15:36)
#
# Please DO NOT delete this file!
# It is necessary for linking the library.

dlname='iconv.dll'
library_names='iconv.lib'
libdir=/home/ama2fr/projects/webti_events/libiconv-1.8/lib

--- libtool.old	2002-08-14 18:33:42.000000000 +0200
+++ libtool	2002-08-14 18:22:16.000000000 +0200
@@ -193,30 +193,17 @@
 old_archive_from_expsyms_cmds="\$DLLTOOL --as=\$AS --dllname \$soname --def \$output_objdir/\$soname-def --output-lib \$output_objdir/\$newlib"
 
 # Commands used to build and install a shared archive.
-archive_cmds=""
-archive_expsym_cmds="if test \\\"x\\\`sed 1q \$export_symbols\\\`\\\" = xEXPORTS; then
-	  cp \$export_symbols \$output_objdir/\$soname-def;
-	else
-	  echo EXPORTS > \$output_objdir/\$soname-def;
-	  _lt_hint=1;
-	  cat \$export_symbols | while read symbol; do
-	   set dummy \\\$symbol;
-	   case \\\$# in
-	     2) echo \\\"   \\\$2 @ \\\$_lt_hint ; \\\" >> \$output_objdir/\$soname-def;;
-	     4) echo \\\"   \\\$2 \\\$3 \\\$4 ; \\\" >> \$output_objdir/\$soname-def; _lt_hint=\\\`expr \\\$_lt_hint - 1\\\`;;
-	     *) echo \\\"   \\\$2 @ \\\$_lt_hint \\\$3 ; \\\" >> \$output_objdir/\$soname-def;;
-	   esac;
-	   _lt_hint=\\\`expr 1 + \\\$_lt_hint\\\`;
-	  done;
-	fi~
-	
-	\$CC -Wl,--base-file,\$output_objdir/\$soname-base -mdll -Wl,-e,_DllMainCRTStartup 12 -o \$output_objdir/\$soname \$libobjs \$deplibs \$compiler_flags~
-	\$DLLTOOL --as=\$AS --dllname \$soname --exclude-symbols DllMain 12,_cygwin_dll_entry 12,_cygwin_noncygwin_dll_entry 12,DllMainCRTStartup 12,DllEntryPoint 12 --def \$output_objdir/\$soname-def --base-file \$output_objdir/\$soname-base --output-exp \$output_objdir/\$soname-exp~
-	\$CC -Wl,--base-file,\$output_objdir/\$soname-base \$output_objdir/\$soname-exp -mdll -Wl,-e,_DllMainCRTStartup 12 -o \$output_objdir/\$soname \$libobjs \$deplibs \$compiler_flags~
-	\$DLLTOOL --as=\$AS --dllname \$soname --exclude-symbols DllMain 12,_cygwin_dll_entry 12,_cygwin_noncygwin_dll_entry 12,DllMainCRTStartup 12,DllEntryPoint 12 --def \$output_objdir/\$soname-def --base-file \$output_objdir/\$soname-base --output-exp \$output_objdir/\$soname-exp --output-lib \$output_objdir/\$libname.dll.a~
-	\$CC \$output_objdir/\$soname-exp -mdll -Wl,-e,_DllMainCRTStartup 12 -o \$output_objdir/\$soname \$libobjs \$deplibs \$compiler_flags"
-postinstall_cmds=""
-postuninstall_cmds=""
+archive_cmds="\$CC -shared \$libobjs \$deplibs \$compiler_flags -o \$output_objdir/\$soname \${wl}--image-base=0x10000000 \${wl}--out-implib,\$lib"
+archive_expsym_cmds="\$CC -shared \$export_symbols \$libobjs \$deplibs \$compiler_flags -o \$output_objdir/\$soname \${wl}--out-implib,\$lib -Wl,--base-file,\$output_objdir/\$soname-base~
+       \$DLLTOOL --as=\$AS --dllname \$soname --exclude-symbols DllMain 12,_cygwin_dll_entry 12,_cygwin_noncygwin_dll_entry 12,DllMainCRTStartup 12,DllEntryPoint 12 --def \$export_symbols --base-file \$output_objdir/\$soname-base --output-exp \$output_objdir/\$soname-exp --output-lib \$output_objdir/\$libname.dll.a"
+postinstall_cmds="base_file=\\\`basename \\\${file}\\\`~
+      dlpath=\\\`bash 2>&1 -c '. \$dir/'\\\${base_file}'i;echo \\\$dlname'\\\`~
+      dldir=\$destdir/\\\`dirname \\\$dlpath\\\`~
+      test -d \\\$dldir || mkdir -p \\\$dldir~
+      \$install_prog \$dir/\$dlname \\\$dldir/\$dlname"
+postuninstall_cmds="dldll=\\\`bash 2>&1 -c '. \$file; echo \\\$dlname'\\\`~
+      dlpath=\$dir/\\\$dldll~
+       \$rm \\\$dlpath"
 
 # Commands to strip libraries.
 old_striplib="strip --strip-debug"
@@ -325,7 +312,7 @@
 # The commands to list exported symbols.
 export_symbols_cmds="
 	\$DLLTOOL --export-all --exclude-symbols DllMain 12,_cygwin_dll_entry 12,_cygwin_noncygwin_dll_entry 12,DllMainCRTStartup 12,DllEntryPoint 12 --output-def \$output_objdir/\$soname-def \$libobjs \$convenience~
-	sed -e \\\"1,/EXPORTS/d\\\" -e \\\"s/ @ [0-9]*//\\\" -e \\\"s/ *;.*\$//\\\" < \$output_objdir/\$soname-def > \$export_symbols"
+	cp \$output_objdir/\$soname-def \$export_symbols"
 
 # The commands to extract the exported symbol list from a shared archive.
 extract_expsyms_cmds="test -f \$output_objdir/impgen.c || \\\\
@@ -3746,7 +3733,7 @@
 	if test -z "$export_symbols"; then
 	  if test "$always_export_symbols" = yes || test -n "$export_symbols_regex"; then
 	    $show "generating symbol list for \`$libname.la'"
-	    export_symbols="$output_objdir/$libname.exp"
+	    export_symbols="$output_objdir/$libname.def"
 	    $run $rm $export_symbols
 	    eval cmds=\"$export_symbols_cmds\"
 	    save_ifs="$IFS"; IFS='~'