Re: g_slice_

["Andrew Paprocki" <andrew ishiboo com>]

Just to put some things in perspective for a moment, I can give a few reasons why I'd support the g_slice implementation. I am definetely not the common glib use case, but it does go to show that the library is used in many different ways.

1) The performance is better than memchunks, which is a given.

2) We do not use linux platforms, and do not use glibc, so reliance on the underlying libc implementation to give you your speed goes against the cross-platform nature of glib. (Similar, in some sense, to gatomic.c's assembler only working in gcc)

3) We write heavily multithreaded applications that have nothing to do with gtk+ but have everything to do with glib providing a stable cross-platform library to allow developers to focus on writing portable c code without having to focus on porting. Apps definetely use GList/GSList/GHash among others in high-performance code where any performance gain is welcomed.

4) Not all environments work in Purify anyway. In our glib implementation, g_malloc != malloc, so we already have to do extra work to make things work in Purify. We also provide an environment setting which enables/disables memchunks for similar purposes.

5) g_slice does what glib does best -- provide a single API that can then be custom tailored under the covers to do what is right. By abstracting the api out, glib could potentially use different internal logic depending on what malloc is being used (decided at configure time). Additionally, if extra information is known about certain platforms that would affect the internal implementation, they can be #ifdef'd.

We were already investigating making a better small object allocator when i heard that g_slice was being checked in, so i am obviously interested in seeing it in action. Everyone should understand that for a lot of people gcc/linux/glibc are not an option, therefore the library should do its best to accomodate all situations.

Perhaps the g_slice code could work through vtables so that Gnumeric/Evolution could simply have the api use their allocator implementation if they felt it was necessary to stay with the existing code. This is a nice feature in the case of the regular gmem functions, so it would be nice to see something similar for g_slice.

Andrew Paprocki
Bloomberg LP

> To: timj imendio com
> From: mortenw gnome org (Morten Welinder)
> Subject: Re: g_slice_
> Date: Mon,  5 Dec 2005 10:44:10 -0500 (EST)
>
> 
> It sounds to me that you had two pieces of code that sucked: the
> glib mempools and the glibc malloc.
> 
> Well known.  Both Gnumeric and Evolution wrote their own specialized
> pool allocators because of that.  (Why two?  Because we didn't know
> about each other.)
> 
> I don't think I should compare slices to memchunks because that is not
> what I have been using.  I have configuring with "--disable-mem-pools
> --enable-gc-friendly" for ages.  Add in using a less bogus malloc and
> you have a better baseline.
> 
> So my working Purify support is now gone.
> So my one piece of hot allocation code is now two.
> So I now have two, not one, allocators keeping seperate free lists.
> 
> You could have avoided the memchunk bogosity by using the malloc-
> wrapping you introduced in early November.  That would have been
> an obvious improvement with none of the downsides.  Improving
> glibc's malloc would probably be a good idea too.
> 
> Which brings me to...  What data structures in glib programs (existing
> and predicted in the next few years) are (a) used multi-threaded, and
> (b) used often enough to warrant performance considerations?
> 
> I see GList nodes, GSList nodes, and GHashTable nodes.  Maybe that leaves
> out one of two, but the scale is right.
> 
> This had better be what you are optimizing for.  Not, for example:
> > - if you use n CPUs (n>1), you want to be able to allocate n times the
> >   amount of memory of a single CPU per time interval, and not just be safe
> >   against concurrent allocations
> > - if you consider one allocator per thread, take into account that chunks
> >   may be allocated in one thread and freed in another
> Those are fine considerations.  Just not for glib.
> 
> Add to that, that the number of C programmers who can write working
> threaded applications is even smaller than the number of C programmers
> who can write code that doesn't leak enough to be a problem.  You are
> not going to see many multi=threaded C applications using glib.  And if
> threading is taking place in a higher language, it isn't clear that
> malloc is where you need the smarts.
> 
> Morten