Re: Asyncronous life helper (Was: EggDBus)

[Alexander Larsson <alexl redhat com>]

On Sat, 2008-12-27 at 15:19 -0500, David Zeuthen wrote:

>  GFuture *future;
>  future = g_future_get (foo->pending_call);
>  g_object_set_data (future, "other-stuff", other_stuff, g_object_unref);
>
> and the GFuture object would be unreffed when the pending call is
> finished.

I'd like to avoid having the data be separately allocated, that lets us
get rid of at least one separate free call/destroy notify. 

> > One major problem about doing things like this in C is that you
> > constantly have to define and allocate small structs to keep the data
> > you need for the completo operation. If we could somehow make this
> > easier that would be a large win. I don't know a great approach to do
> > that offhand though. Some form of user data is one possibility, but I'd
> > really like it to betype safe and without using strings as API.
> 
> If you want type safety you could subclass GFuture and do this
> 
>  subclassed_future = my_subclassed_future_new (...);
>  g_future_set (foo->pending_call, G_FUTURE (subclassed_future));
> 
> but for simple stuff we can have convenience stuff on GFuture just like
> we do for GSimpleAsyncResult.

subclassing is just to much overhead to do per async call. You generally
have lots of async calls in a single sourcecode file, and you can't
realistically use a separate gobject class for each. However, you can
probably define a small struct for each and somehow make a generic
future object contain such a struct.

> > I guess a GFuture/GWish needs to be some form of container that you can
> > grow in size by adding a struct of your own data to it. Maybe we can do
> > it via a macro g_future_push_struct (future, struct LocalOpData,
> > free_op_data) which grows the size of the object and puts the struct at
> > the end. You'll pop it at end to free it.
> > 
> > Of course, such serial/stack-like data management might not be general
> > enought to handle things like parallel async operations. Maybe just a
> > way to add the struct (undefined where) to the GFuture and then you'll
> > get a pointer to the struct inside the GFuture in the completion
> > callback...
> 
> I think a lot of this would be simpler with fibers....

continuations, fibers, threads, etc *vastly* simplify many cases of
async operations. However, they don't express the full solution-space of
async ops. For a simple example, take the sftp gvfs backend, which is
fully async. The get_info() operation does both a stat and an lstat in
parallel and returns when both are completed or on the first error. If
you serialized this via a fiber you'd double the network latency of the
get_info() call. 

Another example is a "perfect" network data copy operation. You need to
keep scheduling both download and upload in parallel so that you always
have data to send. 

So, I think there is a place for a true GFuture-style machinery, even if
you have something like fibers availible. Of course, you'd like them to
be API compatible so that you can use a fiber as a GFuture. For
instance, you might want to express a dependency of your GFuture be the
result from some fiber operation.