[Vala] Implicit lamdas/closures

[Sam Liddicott <sam liddicott com>]

I've been thinking a lot on how vala can make programming asynchronous rpc servers as simple as programming synchronous rpc servers; I.e. No need to worry about continuations/callbacks.

What I've come up with is implicit lamdas; without the need for ()=>{} which is some cases would be nested, gettimg very ugly.

Simple rpc call handlers are self-enclosed and synchronous. Eg

rpc_server_get_next(thing x) {
  return next(x);
}

But some rpc calls can only be satisfied by making other async calls

rpc_server_check_credentials(creds c) {
  req r=auth_server.check_creds(c);
  return r.wait_for_response();
}

And unless you have a new thread or process for each call, this is going to block other rpc requests. Consider this case:

rpc_server_read_file(file f, int offset, int len) {
  return sysread(f, offset, len);
}

It should be possible to use asynchronous io on some modern systems, yet it is hardly worth allocating a new thread over.

Samba4 has each server request marked whether or not it may be processed in an async manner, usually it can.

If the server module chooses to do so it also marks the request when it returns so that a response is not sent right away. The module sends the response later after one or a few callbacks from async client requests used to fulfil the original server request.

However the processing of asyc client responses is generally identical to the processing of the same responses done synchronously.

The general pattern is look this

server_do_things(stuff) {
  req=do_thing(stuff, otherstuff);
  req.add_callback(finish,stuff,otherstuff)
  If (stuff.may_async) {
    stuff.did_async=true;
    Return OK;
  }
  // this also calls finish callback
  req.wait_till_done();
  return(stuff.status); 
}

The obvious candidate for the lambda is the finish function, giving something like this
(assuming lambda local-vars support is complete)

server_do_things(stuff) {
  req=do_thing(stuff, otherstuff);
  req.add_callback( (req) => (
    stuff.status=req.receive(stuff);
    // do more stuff here
  ) )
  If (stuff.may_async) {
    stuff.did_async=true;
    Return OK;
  }
  // this also calls finish callback
  req.wait_till_done();
  return(stuff.status); 
}

It's a little puzzling that the lamda appears halfway through; but it gets more complicated if the server has to issue more than one client request; the nest lambdas make the whole thing look like lisp.

The idea of implicit lamdas is to allow the code to be laid out as if it were synchronous but still work asynchronously, see:

server_do_things(stuff) {
  req=do_thing(stuff, otherstuff);
  req.add_callback(finish,stuff,otherstuff)
  If (stuff.may_async) {
    stuff.did_async=true;
    Return OK;
  } else {
    // this also calls finish callback
    req.wait_till_done();
    return (stuff.status);
  }
finish:
  stuff.status=req.receive(stuff);
  // do more stuff here
  return(req.status); 
}

finish: is a regular C style label; but if it's address is taken for a delegate then it becomes simultaneously a lambda until the end of the enclosing block and also a call to that lambda.

Thus the code can be run_into during normal execution as well as apparently run-into for async execution.

If such a thing were possible, the rpc glue would differ so as to make better use of it.

Here's a more complex case which unwinds a terrible state machine into an apparent linear function:

server_auth(stuff) {
  req=send_get_auth_types(stuff);
  // I'll talk about block this later...
  if (stuff.may_async) {
    req.add_callback(type);
    Stuff.dyd_async=true;
    return;
  } else {
    Req.wait_for_response();
  }
type:
  if (stuff.kerberos) {
    kreq=send_kerberos(krb);
    ...
krb:
    If (kreq.success)...
  } else if (stuff.ntlm) {
  }
}

I said that implicit lambdas ought to last to the end of the containing block, but really when that block finishes a new implicit lambda should start in the outer block as if it were declared in the same manner as discussed; and so on until the end of the enclosing function block is reached.

That way parts of the function can be deferred or executed immediately without a problem.

I don't know how such a lamda would cope with being in a loop.

The If block for which I said: "// I'll talk about block this later..." clearly it's a pain to have to repeat that async/sync fixup code everytime, but I think I've said enough to layout the problem and how lamdas could unwind horrible continuation chains and async state machines, so I'm open for comment.

Sam