Re: [gnet-dev] zero-copy interface?

[Tim Müller <t i m zen co uk>]

On Tue, 2005-11-08 at 07:02 -0500, Jeff Garzik wrote:

> > We could add new API for that, maybe something like:
> > 
> > void
> > gnet_conn_write_direct (GConn* conn, gchar* buffer, gint length,
> >                         GDestroyNotify buffer_destroy_func);
> > 
> > That way you could for example pass an mmap()ed buffer and unmap that in
> > your buffer destroy function when it's been written. Not sure how that
> > compares to sendfile() performance-wise, but it should make quite a bit
> > of a difference in any case.
> 
> The write-completion callback works just fine as a sequence point, 
> indicating GNet is done with the buffer.

I suppose that would do as well, yes. Although it forces you to keep
track of things when you mix different types of buffers. Also, it makes
it easy to implement gnet_conn_write() as

gnet_conn_write_direct (conn, g_memdup (buf, len), len, g_free);

;) I don't really care either way, it was just an idea


> > What's the use case you have in mind for that?  GConn reads into a fixed
> > per-instance buffer, the only expensive thing involved might be a
> > g_memmove() when bytes have been processed, but that you can avoid if
> > you read bytes in any 'denomination' rather than a fixed amount or until
> > the end of the line.
> 
> Several points here:
> * g_memmove() in process_read_buffer() isn't a big deal, since its 
> working on small chunks that are probably in L1/L2 cache.

That's good to know :)


> * The bigger issue is that the app is forced to copy all bytes it cannot 
> act upon immediately, due to GNet's fixed per-instance buffer.
>
> * Also, g_realloc() copies.  Large copies, though certainly it reaches a 
> stable size.

That's why I was asking about the use case scenario for this. I cannot
really imagine a case where this is a huge problem in practice. However,
it shouldn't be hard to add API that replaces GConn's internal buffer
with a user-provided one or to add some kind of 'keep read data until
further notice' mode. 


> The GNet design is pretty optimal.  Just need a few small tweaks.  Need 
> to let the app manage read/write buffers, figure out a good GLib-or-GNet 
> solution for sendfile, and if I'm really lucky, figure out how to do 
> MSG_MORE.

Patches are always welcome ;-)

Cheers
 -Tim