Re: glib memory allocation problems

[Tim Janik <timj imendio com>]

On Tue, 15 May 2007, Dimitrios Apostolou wrote:

> On Tue, 15 May 2007 16:14:26 +0200
> Miklos Szeredi <miklos szeredi hu> wrote:
> > OK, found the problem: g_thread_init() wasn't called by sshfs, and
> > hence the thread private data used by gslice wasn't actually thread
> > private.
> >
> > Now I see that the mandatory use of g_thread_init() is documented, but
> > it's very non-obvious without reading the docs.  And there are no
> > apparent problems if it's not called.  Sshfs got away with it for a
> > long time even though people do stress it pretty heavily.
> >
> > So the question is, should glib not make some sanity checking for code
> > that is not actually thread related (like gslice).
> >
> > Dimitrios, there's a patch attached that should fix the crashes.
> >
> > Thanks,
> > Miklos
>
>
> Thank you Miklos, I am trying the patch now and I'll let you know how it goes. I really hope sshfs won't be crashing again.
>
> I don't understand some things however, so here are some questions to glib devs:
> 1) Why the crash didn't occur when using G_SLICE=always-malloc?

because you used glib invalidly when you used the pthread
API but didn't call g_thread_init. thus, you got crashes
in glib, but not in malloc/free (which is used in case of
G_SLICE=always-malloc).

> 2) I never got any warning when using G_SLICE=debug-blocks, although I reproduced the crash several times. Is it normal?

well, you screwed up glib structures with the threading already,
so you can crash *anywhere* in glib. G_SLICE=debug-blocks just
tells you if you used the g_slice_alloc/g_slice_free API correctly
which you aparently did if you didn't get any warnings.

> 3) What about the constantly increasing memory usage that I described in my first email, is it expected when using G_SLICE=always-malloc?
> I was certainly surprised to witness my system crawl down to a halt, because of swapping...

no, nothing in malloc/free (G_SLICE=always-malloc) or in glib
should constantly eat memory on its own. to have steadily
increasing memory, you need to do something like:
	while (1)
	  {
	    GSList *slist = g_slist_alloc();
	    /* g_slist_free (slist); */
	  }
that'll eventually run out of mem.

i.e. what you describe is most probably something in your code/libs
not releasing memory or ref counts.

> Thanks in advance,
> Dimitris

---
ciaoTJ