Re: [gnet-devel] GNet HTTP server class

[Tim Müller <t i m zen co uk>]

On Wed, 2007-11-28 at 14:13 -0500, Jeff Garzik wrote:

> To that end, here is this afternoon's work, a working embedded HTTP 
> server class, GSHTTP (Gnet Server HTTP):
>  (snip)
> Just like GNet's GServer class, this new GSHTTP class is a thin wrapper 
> on top of GServer and GConn.  It parses an HTTP request with the help of 
> GURI, parses the HTTP headers, and then passes that result to the user's 
> HTTP request function for processing.  The user directly talks to a 
> GConn object from that point on.

Nice! I've also been toying with the idea of writing a simple http
server based on GServer, but haven't gotten around to it yet.  What I
had in mind was a bit more advanced/different though (e.g. adding hooks
for paths/subpaths to serve files directly from disk or call callbacks
to generate content/partial content on the fly and such), something
geared towards simple use cases with few clients rather than something
really scalable.

> I have attached ApacheBench results on a gigabit network between two 
> fast computers, for 1,000,000 requests of a 4K file.  The results are 
> decent for a couple hours of work, but not great.  There are WAY too 
> many per-request memory allocations and copies.  As a result, I was 
> actually bounded not by network bandwidth but CPU usage -- the app was 
> consuming 100% CPU (it is single-threaded) during the ApacheBench run.

Interesting.  Do you have a breakdown of where it spends most time?


> Some of that is the app's fault, but some of that is GNet.  In 
> high-volume server applications, the only data copies that take place 
> occur when calling system calls (read, write).  The server app _never_ 
> copies data, if it doesn't have to.  Ditto for allocations.  That was my 
> motivation for adding gnet_conn_write_direct(), and I am thinking about 
> adding gnet_conn_read_direct() as well.

Yes, I was also thinking about something along the lines of
_read_direct() recently.  Also, some sort of _write_direct_full() with
an additional user_data argument might be useful too (e.g. for cases
where you have a refcounted FooBuffer struct where you want to write
buffer->data, but then unref the buffer itself when done).


> GURI does too many string copies and allocations to be used on a
>  per-HTTP-request basis, too, IMO.

Could be, I doubt the code was written with efficiency in mind, although
I don't really see how you'd easily get around all the strdups for the
different bits and pieces (unless you pass in a writable string and let
gnet munge bits in the middle into terminators maybe?).  Maybe it's just
not the right tool for the task.  In any case, it should be fairly easy
to benchmark its performance to see if there's some low-hanging fruit,
no?

I'm wondering whether the GLib main loop is really suitable for handling
LOTS of file descriptors/clients efficiently (well, only one way to find
out I guess ...).  There's a patch to make it use epoll in bugzilla [1],
but it doesn't look like that's going to go in any time soon.

Cheers
 -Tim

[1] http://bugzilla.gnome.org/show_bug.cgi?id=156048