Glib: a Win32 discussion

[Kean Johnston <kean johnston gmail com>]

Everyone,

WARNING: long, detailed message. If you don't care about Win32, move on.

I would like to start a discussion on making changes to Glib for improved 
Win32 support. These changes will eliminate many of the pitfalls that 
usually accompany that platform, and better live up to the "mission 
statement" of Glib (which isn't really a mission statement, but instead is 
an indication to users of what it is trying to achieve - "It works on many 
UNIX-like platforms, Windows, OS/2 and BeOS". These are just ideas, and I 
appreciate any and all feedback.

Before even discussing the changes I'd like to propose, it is important to 
discuss the actual problems the changes are trying to solve. I am not 
making any changes just for change's sake. First and foremost of those 
changes is the complete and total avoidance of the C runtime DLL, 
msvcrt.dll. There are many good reasons for doing this. First, none of the 
modern MS compilers allow you to directly target using msvcrt.dll (which is 
present on all Windows systems) and instead force you to use 
compiler-specific versions of the C runtime, which you then have the 
obligation to either install yourself, or ensure are installed. It is 
possible to craft a set of tools (using a mixture of the driver development 
kit, platform development kit, Visual Studio and a large number of hacks) 
that will allow modern MS compilers to target msvcrt.dll, but doing so is 
extraordinarily complicated and one tiny mis-step along the way causes 
things to fail in subtle ways. An alternative is to use and support only 
MinGW and MSYS, and this is really tempting, except that using these tools 
you can NOT avoid dependence on msvcrt.dll. There are severe limitations 
imposed by using it, not least of which is the fact all file descriptors, 
the heap and other information is local to each DLL. Unlike the UNIX world 
where a shared object uses the same malloc as an application (under almost 
all circumstances), an object created in one DLL cannot be freed by 
another, or a file descriptor opened in one cannot be closed in another. 
The only way to ensure this does not happen is to guarantee that every 
single library you build with uses the exact same CRT, and that it is 
shared. This is very difficult to achieve.

By avoiding the CRT altogether and using native Windows functions that use 
handles, and using a memory allocator other than malloc, all of these 
problems go away. Glib already makes this possible by providing its own 
malloc wrappers, but the current Win32 implementation is the same as the 
UNIX one in that it is just a very thin layer on top of malloc. The first 
thing that should change is making g_malloc and friends use the HeapAlloc 
function, and ensure that g_mem_is_system_malloc() always returns false. 
This is really easy to do and shouldn't upset the apple-cart too much.

Memory allocation is the easy bit. HeapAlloc maps nicely onto malloc. There 
are other bits that are more problematic. The next easiest to solve is 
stat(). Gio already makes a token gesture at wrapping stat() but its 
wrapping is a bit TOO thin. Not only is the stat structure problematic 
under Windows, it is problematic under UNIX too, as many stat structures, 
even directly out of sys/stat.h, change according to pre-processor defines 
(LFS). Ideally, we should create a GStatBuf structure that is defined only 
in terms of portable data types that are like-sized on all systems. We 
should ensure that the file sizes are always 64 bits, and that the 
timestamps are always 32-bits, using 1-1-1970 as the epoch (standard UNIX 
time). Bonus points if we provide *both* 32 and 64-bit times. If we did 
that, we would need to come to some agreement on exactly what a 64-bit time 
represents. Then, we ensure that g_stat() always fills in GStatBuf with 
whatever conversions are necessary, so that a unified interface is 
presented to an application. This too, is very easy to do and shouldn't 
upset the apple-cart too much, with the exception of 64-bit time stamps, 
which, if people want to support, may take some hashing out. Wrapping stat 
can also fix the problem of having the uid_t / gid_t types differ not only 
from system to system, but also change on the SAME system depending on 
defines. We can force these to always be 32-bit. Similarly, ino_t.

The biggest and most challenging thing I would like to propose is the 
addition of gstdio. Currently there is no attempt to provide a portable 
standard IO set of functions, and these are notorious for subtle 
differences. For example, does snprintf() return the number of characters 
that would be required if N were unlimited? We need not develop a stdio 
package from scratch, not by any means, but can instead base it off glibc's 
stdio, which is also LGPL'ed. We can modify the internals when necessary to 
use glib functions (for example, use g_malloc instead of malloc). This 
package can be easily modified to return a Glib-abstracted FILE structure 
(GFILE). On systems where stdio is known to have a certain behaviour this 
whole package can be a very thin wrapping, but the behaviour of all of the 
functions should be identical on all platforms Glib supports. Obviously 
fstat() would become g_fstat() and conform to the unified stat structure 
mentioned above.

Last, but by no means least, is the reliance on "compiled" files, like 
compiled schemas (or in the case of Gtk, icon caches). On UNIX systems 
where things are installed in a universally-accessible location, this isn't 
a problem, but on Win32, where multiple applications could all include 
their own private copies of the DLL's, this is a problem. Fixing this is a 
bit tricky but very doable. Windows does provide two places that are 
predictable and universally accessible: the registry, and %ProgramData%. 
The registry is a poor choice except perhaps for location files inside 
%ProgramData%. The registry is slow, and also imposes some severe 
limitations on key sizes etc. This can be very easily addressed by 
compiling Glib with LIBDIR set to something like "%ProgramData%\Glib\2.x" 
and ensuring that functions like g_file_get_contents() or g_open() all call 
ExpandEnvironmentStrings() on Win32. This is also a relatively small change 
and doesn't change any existing API's (although on Win32 it will have a 
behaviour change).

I don't know if there has been a discussion on a Glib 3.0, but perhaps all 
of this could be the basis for one (especially with the addition of 
gstdio). I am volunteering to spear-head all of this work. I don't have 
write access to anything but if there isn't already a 3.0 branch, and 
people like / agree / support the above, perhaps we can change that and I 
can start work.

One last thing, since it has proven to be a source of considerable 
incompatibility, and that's the reliance on D-Bus. I think it should remain 
possible to use dbus if you want it, if your application really needs it, 
but to have relatively (from my position of ignorance) unrelated things 
like gapplication absolutely rely on it is a mistake, as far as I can see. 
dbus is really not appropriate for, or required for, many applications that 
would otherwise want to use Glib/Gtk. Should my Windows MP3 editor really 
need dbus daemons running just because it uses GtkApplication (which in 
turn uses gio/glib and thus the dbus coupling)? Seems like a good bit of 
gio could use some re-thinking and this again is maybe more appropriate for 
a 3.0 release where we may be able to get away with slightly more 
disruptive changes.

Thank you for your time reading this, and I welcome comments and debate.

Kean