Re: Updating GUI during long operation

[Kip Warner <kip thevertigo com>]

On Fri, 2013-04-26 at 15:16 +0200, Colomban Wendling wrote:
> That's weird.  However, although I don't know much about Python
> threading (and another guy in this thread suggests it's not really
> good), I'm wondering whether your thread couldn't be simply locking
> stuff used by both GUI and worker thread, effectively blocking the GUI one.

It's possible, but I'm not explicitly trying to lock any resources (e.g.
via a mutex or what have you).

> For example, your idle callback _updateGUI() uses the thread's self, so
> I guess Python would lock it.  

Ah, I see what you mean (GIL).

> And this would mean that the callback and
> the thread functions could not run concurrently.  And even worse, if the
> Python thread model is based on simple functions locks (e.g. the lock is
> held at the function entrance, and only released on function leaving),
> this would mean the callback could never run before the thread as
> finished.  I don't know, but this is an idea from where the issue might
> come from.

Yes, that's possible. But the good news is that when I took Simon's
advice and replaced the GObject.idle_add() calls with ones to
Gdk.threads_add_idle(), the GUI seems fully responsive.

> BTW, in VerificationThread.run, which I believe is run in the thread,
> right?, there are access to GTK widgets (self._assistant).  This either
> requires a lock (which may be the reason why your thread would block the
> main one), or it may crash (since GTK is NOT thread safe).  Anyway, I
> think this should rather be done in the idle callback.  BTW, you
> actually mark the page as complete in both run() and _setQuitOk() which
> is called by run().

Yes, you're right. Those calls were only made at the end of the thread's
execution, but they shouldn't be there anyways and should have been
moved into the idle callback. Thanks for spotting that.

> I don't think optimizing the worker thread is the solution for the GUI
> issue.  If everything worked as it should, the worker thread could take
> as much time as it wanted, and it still wouldn't block the main thread.
>  That's the point of threads: doing tasks in parallel.  Of course
> reducing the time the worker thread takes would reduce the UI freeze
> duration, but it wouldn't solve it.

I agree and in a perfect world, that's what would happen. Except I
think there are peculiarities, as I am learning, in how the GIL is
implemented as well as subtle things in how animations are driven with
the Gtk Image widget. Thankfully it seems to work fine now (today, at
least) with the conversion to Gdk.threads_add_idle()

> However, if most of your thread's time is lost in IO rather than
> computation time (e.g. if the MD5 runs faster than the IO), you could
> perhaps avoid a thread using asynchronous IO calls and computing the sum
> by packets.  Something like this (in Vala pseudo-code):
>
>       async string sum(File file) {
>               var sum = new MD5();
>               while (true) {
>                       /* read by blocks of 2048 bytes (or whatever) */
>                       var data = yield file.read_async(2048);
>                       if (data) {
>                               sum.push(data);
>                       } else {
>                               break;
>                       }
>               }
>               return sum.hash();
>       }
>
> Here yield would work like in Vala, which basically means calling a
> implicit callback which will perform the following instructions when the
> asynchronous work is done.
>
> But this doesn't explain why your program blocks the main thread, and
> may very well not be practical in your case (or even, on Python, I don't
> know).

I thought of the asynchronous approach and it's possible, but I am lazy
and don't really want to have to re-factor a lot of code now. But
thankfully it seems to work better now anyways. But I'll definitely keep
that approach in mind in the future.

> Hope this could help in some way.  Regards,
> Colomban

Thanks a lot Colomban. Like before, your thoughts were very constructive
and useful.

-- 
Kip Warner -- Software Engineer
OpenPGP encrypted/signed mail preferred
http://www.thevertigo.com