Re: GThreads and GThreadPool help for a dummy

[richard boaz <ivor boaz gmail com>]

I realized after i sent that email that it was a little rushed, and too unqualified.  So in the interest of completeness (and to avoid possible misuse), the following addendum to the last post:

The code provided assumed a background program, i.e., not a GUI or a program running with a main loop, since the call to g_async_queue_pop() in the main routine is blocking.  As written, this code should absolutely NOT be used in a program where a main loop is expected to remain responsive.

However, with a little modification, the algorithm can easily be made non-blocking and usable in a GUI:

• Remove all references to the async queue commQ.
• In finishThreads(), instead of returning the answer via commQ, use g_idle_add() to pass the result back to the main loop.  (write a new routine that takes the result and presents it back to the user, whatever that means.)

In general, too, the following warnings must accompany this code:

• Case is not handled when code is called a second time before the first call has completed, i.e., it is not re-entrant as written.  Calling a 2nd time before 1st execution has completed will cause indecipherable havoc; either handle this case, or disallow it.
• There are memory leaks.

And, while I'm here, a couple of relevant guidelines to writing multi-threaded code:

• Use as few locks as possible in the threads.  In the example provided, the stored result could have been referred to directly (added to) in each executing thread, using a lock around the summation.  Instead, the solution provides for each thread to access to a single (and unique) element of an array of float values holding the summing result, while the finishThreads sums each thread's result once all threads have completed, returning this single value to the main routine/loop.  In this manner, no lock is necessary here in computing the result, guaranteeing a faster completion.

• Whatever code must exist inside a lock, make it absolutely as small as possible.  The more code that exists inside a lock, the greater the chance that other threads will get stuck waiting on the lock to be free.  The more threads that must sit and wait for a lock to be free, the less the point of having made the code multi-threaded in the first place (since, by definition, locked code means only one can execute at a time, not much different than writing in-line...).  In the provided solution, a single lock is required around g_queue_pop_head() (which executes very fast), making the likelihood of a thread having to wait on the lock to be free very low indeed.

cheers,

richard

2010/7/23 richard boaz <ivor boaz gmail com>

this is how i do it (not using pools), i have modified my code for your purposes, though it's probably wrong somewhere in the implementation details (i didn't even compile it), but i leave that to you.

this is very easily generalizable to do any kind of background work on a "list" of items.

richard

=== BEGIN code snippet ===

{ // in the main routine

gfloat *retPtr, tot_err;

GQueue *threadsQ = g_queue_new(); // Q holding list of data items to process

data_t data[N_DATAPOINTS]; // data items to process

fill_the_data_array(data);

for(i = 0; 

i < N_DATAPOINTS;

i++)

{

g_queue_push_head(threadsQ, &data[i]); // put data item onto Q for processing

}

g_thread_init(NULL);

commQ = g_async_queue_new(); // create comms Q 

threadsProc(threadsQ, maxThreads); // initiate threads

retPtr = (g_float*) g_async_queue_pop(commQ); // sit and wait for all threads in threadsProc() to complete...

tot_err = *retPtr; // the answer

}

// all the rest in a different source file...

static GQueue *threadsQ; // Q holding the data to process

static int numThreads; // number of threads (and a counter too)

static g_float *tot_err; // array of g_float, one element per thread

static g_float total_errors; // total errors, added up, returned to main

static GStaticMutex QMutex = G_STATIC_MUTEX_INIT; // mutex to read Q

typedef struct _THREADARGS

{ // structure of arguments to thread proc

GThreadFunc returnFunc;

int threadNum;

} THREADARGS;

static void finishThreads()

{ // called once a thread has finished its work

numThreads--;

if (!numThreads)

{ // return to main routine

int i;

g_float retValue = 0;

for (i = 0;i<numThreads; i++)

retValue += tot_err[i];

total_errors = retValue;

g_async_queue_push(commQ, &total_errors);

}

}

static data_t *get_data()

{ // get the next data item off the Q to process, called by _addData()

data_t *data;

g_static_mutex_lock(&QMutex);

data = "">

g_static_mutex_unlock(&QMutex);

return (data);

}

static void _addData(THREADARGS *threadArgs)

{ // processing thread

int threadNum = threadArgs->threadNum;

GThreadFunc retFunc = threadArgs->returnFunc;

data_t *data = "" style="white-space:pre-wrap"> // get a data item to process

while (data)

{

tot_err[threadNum] += calc_error(data);

data = "" style="white-space:pre-wrap"> // until no more to do

}

(*(retFunc))(NULL);

}

void threadsProc(GQueue *q, int nThreads)

{

int i;

numThreads = nThreads;

tot_err = calloc(numThreads, sizeof(g_float));

threadsQ = q;

for(i=0;i<numThreads;i++)

{

THREADARGS *threadArgs;

threadArgs = calloc(1, sizeof(THREADARGS));

threadArgs->returnFunc = (GThreadFunc) finishThreads;

threadArgs->threadNum = i;

g_thread_create((GThreadFunc) _addData, (void *) threadArgs, FALSE, NULL); 

}

}

=== END code snippet ===

2010/7/23 Øystein Schønning-Johansen <oystein gnubg org>

I tried, and the new problem is now:

 pool = g_thread_pool_new( my_calc, (gpointer) &common, 8, FALSE, &err );

 for (i = 0; i < n_tot; i++ )
         g_thread_pool_push( pool, &store[i], &err );

 g_thread_pool_free( pool, FALSE, TRUE );

When I run this code, it continues without waiting for the threads to
complete. I expected that the g_thread_pool_free function would make it wait
for all threads to complete before the program code continues, however
it does not! How can I make it wait?

(Yes, I added a static mutex in my_func. I'm convinced that I need it. )

-Øystein
_______________________________________________
gtk-list mailing list
gtk-list gnome org
http://mail.gnome.org/mailman/listinfo/gtk-list