gconfd process eating CPU

[Brian Cameron <Brian Cameron Sun COM>]

I am working closely with the SunRay group and we have discovered that
in a rare situation, the gconfd-2 process seems to fall into a polling
loop that ends up eating 100% of the CPU.  Typically, a gconfd-2 only
uses less less than 0.1%.  While it is rare for it to fall into the
polling loop, it does happen frequently enough that it is causing
problems on our multi-user servers.  Typically our multi-user servers
have 4 or more processors, but loosing one processor due to a runaway
process has a huge performance impact.

While we do not know how to recreate the problem, we have a mechanism
for identifying machines that have runaway processes on them, so it is
no problem for me to find a machine running a sick gconfd-2 process.
Note that we set the "ORBIIOPIPv4=1" in our /etc/orbitrc file, so we
are using network aware gconf.

Such sick gconfd-2 processes have the following stack trace:

  core 'core.64457' of 64457:     /usr/lib/gconfd-2 13
   ff19d9c8 _poll    (39660, d, 7055, 56240, 0, 3) + 8
   ff22a28c g_main_context_poll (35e20, 7055, 7fffffff, 39660, d, ff280410) + d4
   ff2294cc g_main_context_iterate (35e20, 1, 1, 0, 35e28, 39660) + 37c
   ff229e00 g_main_loop_run (3dfd8, 1, 0, ff282c54, 2c3f0, 400) + 300
   000167c4 gconf_main (3dfd8, 17400, 2a000, 2ad58, 0, 2ac2c) + 90
   00016508 main     (0, fbc9, 2abd4, 0, 18fe8, 33b88) + 4d8
   000130d0 _start   (0, 0, 0, 0, 0, 0) + 108

Unfortunately, that doesn't really give me much of a hint of what is going
wrong with gconfd-2.  I'm hoping someone might be able to help suggest some
ways that we could figure out what might be going on with gconfd-2.  I'm
working with the sysadmins on the machine where gconf is hanging to turn
on syslog debugging for gcond-2 so we can get additional debug information
that might highlight what gconfd-2 is thinking it is polling for.

I have core files from processes that were running in a bad state if they are
at all useful for debugging.

--

Brian