Re: simplifying closures

[Havoc Pennington <hp redhat com>]

Hi,

Before diving into the old thread, Owen points out that it doesn't
even make sense to have multiple data objects in C, because you have
to write a marshaller if you want to pass multiple pieces of data to a
callback.

The only way to get multiple data is by following the example you gave
a couple posts back; create a struct with two GObject, explicitly add
both GObject as dependencies. Then the destroy notifier frees the
struct, but not the objects; and you can't hold a ref to the
objects. You also have to add the pre/post callbacks or it will all be
unsafe. That's much too hard to understand for anyone to end up using
it in C. People don't understand ref/sink and floating refcounts, even
lots of hackers that I'd consider competent. Introducing a notion of
"explicit dependency" is just not going to fly.

Karl Nelson <kenelson ece ucdavis edu> writes: 
> Yeah, I think you definitely started your arguments in the wrong 
> foot.  You attacked the premise on the basis of memory where I
> am afraid chaining always loses because every time you need to chain 
> your memory size doubles.

The reason I brought up memory is related to a conversation I was
having with Tim in person, it wasn't meant to be the main thrust of
the argument.

However; I don't think chaining results in memory size doubling in the
language binding case.

The reason is that a language binding typically needs its own chunk of
memory anyway. e.g. in Guile you have a scheme function which you have
to store someplace. So you would already need an object to store that
function, and you can just have that same object be the proxy you
chain to.

If you create huge chains, then sure, but we aren't going to.  It has
the same problems as supporting the huge dependency graph in the
closure itself.

>  It also creates a very large web of dependencies
> in the chains with becomes fragle and hard to deal with.
>

I don't think so. I think you'd normally have a chain to one proxy
that contains the e.g. Scheme function to call.
 
>   1) Make LB less of second class citizens.

I wouldn't phrase it that way; it's just vague and meaningless. The
point is to make language bindings work properly by adding _full
variants everywhere, and make it easy for libraries to implement the
_full variant.

>   2) Give the same separation of connection and the flexiblity it
>      allows to the C programmers.
>      - gtk_signal_connect and varients exist in gtk+ for historic
>        reasons.  
>      - closures are the mechanism of glib.
>      - C programmers learn closures,
>      - closures are equal in every way to a function pointer.
>        (copyable and resable)
>      - closures should meet most practical needs for their concept.
> 

I'm not convinced that any of this is needed.

> Now I would argue that to meet number 1, your really have to suceed
> well in number 2.  That is that if the closure API isn't good enough
> for the C programmers to use then they won't use it.  And since 
> LB need the closures they need the API's to think in terms of closures
> otherwise LB will find the vast majority of the structures using
> and passing around function pointers.  Thus the quality of the API
> if closures aren't useful to C programmers will be low to LB.
>  

I disagree. Language bindings work fine wherever we have a _full
variant. Therefore, the _needed_ function of the closure is to ensure
we always have a _full variant. That's it. Anything else is
convenience functionality.
 
> However, if closures became a primary concept for both C and LB then
> I could write the API like this....  (not totally correct, but
> you get the picture.)
> 
> 
>    GtkXML* xml=gtk_xml_new();
> 
>    /* set up a pool of symbolic closures to bind to */
>    gtk_xml_add("open",g_closure_new(&function1,data1));
>    gtk_xml_add("help",g_closure_object_new(help_dialog,&gtk_widget_show));
>    gtk_xml_add("close",g_closure_new(&function3,data3));
>    gtk_xml_add("new_file",g_closure_full_new(&function1,
>                                              g_strdup("file"),
>                                              &g_closure_free_data));
> 
>    /* load the xml which binds the names symbolicly */
>    gtk_xml_load("my.xml");
> 
>    /* connect remaining closures using C magic */
>    gtk_xml_autobind();  
>
... 
> Now if there were 5 widgets which could send "open" they all get
> hooked up to the same closure.

The feature you need to do this is connecting one closure to multiple 
parents.

But a trivial alternative implementation is:

 gtk_xml_set_callback_lookup_func (xml, callback_lookup_func);

Where callback_lookup_func() will take care of looking up the closure
for a given name. It will create a new closure each time from a native
function object. Works fine.

Now you don't need multiple parents anymore.

> Generally I agree here.  Language bindings can live with the 
> ( 1 invalid, 1 dtor) system.  However, this means the C widget
> writer must know that the hell the internals are and at the
> same time must know how to sucessfully chain them.  I can see
> the fun if someone assumes they can use some internal and then
> that internal gets taken by the language binder.  Allow an arbitrary
> number and the users doesn't need to peek in to see if some
> else has used the resource up.
>

In each context, the inotify is defined to be the parent of the
closure (the signal connection, etc.). The destroy notifier is always
for the application writer, not the library writer, in C; and for the
language binding, not the application writer, in language
bindings. Those are the rules we have now; they work fine.
 
> > I'd like to start over on the closure thing, and from an
> > end-programmer perspective, see convincing evidence that someone will
> > use these features.
> 
> Start over on the closure implementation or the discussion? 
> Who will be the judge the merits?  You?  Me?  Tim?  Considering
> my ablity to convince you of anything I think I would rather take
> chances with a firing squad. ;-)
> 

Start over on the discussion. Owen and Tim are the judges of the
merits. At the moment Owen seems to dislike the complex closure
(that's why I posted this mail), so we need to work through the
issues.
 
> You argument of why we would never use it in C is just like that when I 
> proposed libsigc++ to Qt.  They said we have a simpler system which 
> only allowed direct connections.  They looked at the code base and don't 
> see where it would get used as anything else.  What the missed is all
> the places where the implemented a complex API to get around the limitation
> and thus the limitation is no longer apparent.
> 

So my question: where is this complex API in GTK+?

> Time and again I look at what people do with gtk--/libsigc++ code
> and I am surprised.  They build event queues and multitheading systems.
> They wanted to copy them.  At the same time I have found in gtk-- code,
> it openned up a whole basket of things which made the design simpler I 
> didn't see when I created the system.  It solved may of the problems
> I have previously considered impossible.
>

FWIW, I can't imagine keeping all this closure stuff sane in a
multithreaded event queue. ;-) It's bad enough as-is.

Havoc