Re: Updating GUI during long operation

[Simon Feltman <s feltman gmail com>]

On Sat, Apr 20, 2013 at 4:25 PM, Kip Warner <kip thevertigo com> wrote:

>   <http://rod.gs/Sio>


A brief look at the source and it seems there is nothing calling
Gdk.threads_init which I think is needed. Similarly, use
Gdk.threads_add_idle instead of GObject.idle_add for scheduling GUI updates
from worker threads. The necessity of these is still somewhat unclear to me
at the moment though.

Enough questions have come up recently with Python and GTK+ threading that
I've added a wiki page here:
https://live.gnome.org/PyGObject/Threading

This is very much a work in progress and I appreciate updates or validation
if you find any of this useful.

The threading model in Python is pretty bad and you might consider a
different technique (please read up on the Python GIL if you don't already
know how threading in Python works). If the operation is IO bound, look
into using async IO with Gio. If it is CPU bound, take a look at the Python
"processing" module or even numpy for certain things. These techniques will
make the app much more responsive because other threads won't be blocked by
the GIL.

A surprising example is as follows:

> > > sum(range(1000000))

This seems to completely block any other Python threads for the entirety of
the operation. I think the reason is because the GIL is held the whole time
and Python is not giving time to other threads for the duration of the call
(this includes GTK+ update callbacks written in Python).

If you split the operation up, you can gain interactivity with other
threads because Python will interleave statement execution with giving time
to other threads (at the cost of performance).

> > > total = 0
> > > for i in range(0, 1000000, 100):
...     total += sum(range(i, i + 100))

In this sense Python is really always single threaded in regards to running
pure Python code in which none of the statements ever release the GIL (IO
related functions should be fine though). And if there are not any calls
which release the GIL, Python threads might just be adding overhead and
confusion.

Finally, the same example using numpy will allow the same operation to run
completely unblocked by the GIL in its own thread, so you keep full
interactivity with actual concurrent processing of the threads (and the
operation is about 10 times faster than the first example).

> > > numpy.sum(numpy.arange(1000000))


-Simon