Re: Changes to the GLib main loop [ g_main_iterate() ]

[Owen Taylor <otaylor redhat com>]

[ Breaking up this reply into multiple pieces ]

Sebastian Wilhelmi <wilhelmi ira uka de> writes:

> > > For multithreaded programs there is the added advantage, that you can
> > > use the usual idioms in other threads as well (though they will do
> > > nothing then, just wait, whenever may_block is true.):
> > >
> > > while (g_main_context_pending(context))
> > >   g_main_context_iteration(context, may_block)
> > 
> > But the way you've written it, this doesn't do at all what people
> > expect.
> > 
> > What people will try to use this for is:
> > 
> >  - Main thread is blocking in a long computation
> >  - To continue processing events, iterate the main loop from
> >    a different thread.
> 
> I don't think the programmer expects that. I think
> 
> while (g_main_context_pending(context))
>   g_main_context_iteration(context, may_block)
> 
> is a idiom for saying "if there are any pending updates for e.g. GTk windows,
> do them, before I continue a long computation". 

Hmmm, so what you are saying is that you think that this idiom,
preferably written as:

  while (g_main_context_iteration (context, FALSE))
    /* nothing */;

Is saying "if nobody is handling events currently, then handle anything
pending." And the consider if any instances of g_main_run() are running,
someone is handling events.

This doesn't correspond to experience - every time that someone has 
asked about iterating the main loop from the other thread, they have
been interested in handling GUI events while the main loop is blocking.

Now it may be the case that having g_main_context_iteration (context, FALSE);
do nothing and return FALSE if the context is owned elsewhere is
the best we can do and better than what we currently do - which
is to warn and do nothing. But I don't think it what people expect
or want.

My other fear would be that people would write:

 while (i_dont_have_a_message)
   g_main_context_iteration (context, TRUE);

Which, in a non-owner thread is hideously expensive. Just like the
case of the g_main_context_run(). In some sense, what we need
is:

 g_main_context_wait (GMainContext *context, GCond *condition);

Which waits by a combination of g_cond_wait() and g_main_iteration()
until the condition is broadcast. Of course, this isn't really
implementable, at least without hacking GCond to pieces. And
actually, g_main_context_run() is already means something a bit
like this.

> And indeed these updates are done, but in another thread, so FALSE
> is returned. and the loop is not entered. Of course the other thread
> might block itself in a long computation, but again it might call
> the above two lines from time to time and it still works.

I'm not sure what the definition of "work" is. If you mean "doesn't
produce warnings", then sure. If you mean, "cause anything useful
to be done", I don't see how it does that. 

The common case is that one thread is inside a g_main_loop_run() for
the entire execution time of the program. Either that thread
is already processing events, or it is blocking in a computation.
Either way g_main_context_iteration() in another thread doesn't
do any good.
 
> Whenever some thread "owns" the context, I think there is no
> sensible way to poll and dispatch on that context from another
> thread, e.g. trying to continue processing events while another
> thread blocks while dispatching that events.

Hard, definitely hard. I'm not sure I'd say "no sensible way".
But, yes, there is a reason why I didn't implement this.

> > But that doesn't work since g_main_context_iteration() will just wait
> > for the main thread. So allowing someone to write this is just
> > misleading them.
> 
> g_main_context_iteration() is blocking, but g_main_context_pending is
> returning FALSE. It's just as it would happen in a single threaded program,
> only that g_main_context_iteration wouldn't do anything, as everything is
> already done in another thread, not very surprising, I think.

Past experience says that it _is_ suprising that g_main_iteration() 
in another thread doesn't handle events in that other thread when
the main is blocking.

So, in summary:

 * I don't see (with your scheme) g_main_context_iteration() from
   another thread as being useful, though g_main_context_run()  
   is.

 * However, if making g_main_context_iteration() from another
   thread not warn is the price of enabling:

    - Moving a loop between threads
    - Allowing creating a loop before the thread
  
   Then perhaps we should do that, and just be prepared for
   some questions on gtk-list ;-)

> > If we actually want to allow people to call g_main_context_iteration()
> > from multiple threads, then we have to implement it so that if one
> > thread is stuck in dispatch() then another thread will recursively
> > iterate; if the first thread is however, blocking in poll(), or is
> > doing anything other than dispatching, then it has to simply
> > wait.
> > 
> > This is conceptually possible, but quite a pain to implement, which is
> > one reason I didn't do it. (Remember, the context lock is _not_ held
> > over prepare/query/check.) There are also questions of what happens to
> > outstanding dispatches when multiple threads are iterating on the main
> > loop. A recursive invocation from a single loop simply discards
> > outstanding dispatches and starts over, but this clearly isn't the
> > right behavior for multiple threads.
> > 
> > Possibly the right behavior is for the threads to dispatch requests
> > round robin, and for the first free thread to start a new cycle,
> > ignoring sources currently being dispatched. This would need
> > _very_ careful definition, and the combination of this and
> > recursion within threads presents a highly complex situation.
> 
> That is true, it sound like a implementation nightmare with no real benefit.
> If the user want's to dispatch certain events to other threads, s/he should
> use GAsyncQueue or better yet GThreadPool, which is just for doing that.

I'd tend to agree that the structure should be:

 * One thread handles incoming events, and dispatches all slow operations
   to other threads.

It's not the only way people try to write things, but probably we
just need to educate people.
 
Regards,
                                        Owen