Re: [gnet-devel] GNet HTTP server class
- From: Jeff Garzik <jeff garzik org>
- To: Tim Müller <t i m zen co uk>
- Cc: gnet-devel-list gnome org
- Subject: Re: [gnet-devel] GNet HTTP server class
- Date: Fri, 07 Dec 2007 20:50:49 -0500
Tim Müller wrote:
On Wed, 2007-11-28 at 14:13 -0500, Jeff Garzik wrote:
To that end, here is this afternoon's work, a working embedded HTTP
server class, GSHTTP (Gnet Server HTTP):
Just like GNet's GServer class, this new GSHTTP class is a thin wrapper
on top of GServer and GConn. It parses an HTTP request with the help of
GURI, parses the HTTP headers, and then passes that result to the user's
HTTP request function for processing. The user directly talks to a
GConn object from that point on.
Nice! I've also been toying with the idea of writing a simple http
server based on GServer, but haven't gotten around to it yet. What I
had in mind was a bit more advanced/different though (e.g. adding hooks
for paths/subpaths to serve files directly from disk or call callbacks
to generate content/partial content on the fly and such), something
geared towards simple use cases with few clients rather than something
That would be pretty easy to add to GSHTTP.
I had been thinking about a way to register a list of callbacks, each of
which was associated with a static string or GRegex, to be matched
against each incoming URI.
Then, like Apache's core module infrastructure, call -each- matching
callback, until the end-of-list or a callback returns a status
indicating that no further processing of that URI should occur.
It would _really_ be nice if we had a GLib-ish portable version of
sendfile(2), which almost every OS (including Windows) supports. That
would help facilitate your suggestion, having a GSHTTP helper function
for serving files off disk. Design the interface correctly, and an
OS-specific sendfile could be used if available, or regular file
operations used if not.
I have attached ApacheBench results on a gigabit network between two
fast computers, for 1,000,000 requests of a 4K file. The results are
decent for a couple hours of work, but not great. There are WAY too
many per-request memory allocations and copies. As a result, I was
actually bounded not by network bandwidth but CPU usage -- the app was
consuming 100% CPU (it is single-threaded) during the ApacheBench run.
Interesting. Do you have a breakdown of where it spends most time?
After GSHTTP, I wrote an epoll-based, non-GLib/GNet server. It
allocated a static, writable 8K buffer to each connected TCP client. It
was fairly straightforward to parse the URI and headers by overwriting
end-of-line markers with nuls.
Another thing that probably hurt my GSHTTP implementation was copying
each header, key and value, into a GHashTable. As I did in the non-GLib
version, it is better to store a pair of pointers to C strings than copy
all the headers.
GURI does too many string copies and allocations to be used on a
per-HTTP-request basis, too, IMO.
Could be, I doubt the code was written with efficiency in mind, although
Actually it's pretty darn efficient, if you replace each g_strdup() with
an in-situ nul termination. (I did this in my non-GLib HTTP server)
I don't really see how you'd easily get around all the strdups for the
different bits and pieces (unless you pass in a writable string and let
gnet munge bits in the middle into terminators maybe?). Maybe it's just
I'm wondering whether the GLib main loop is really suitable for handling
LOTS of file descriptors/clients efficiently (well, only one way to find
out I guess ...). There's a patch to make it use epoll in bugzilla ,
but it doesn't look like that's going to go in any time soon.
Note my name in there ;-)
GLib main loop isn't as nice a epoll... but it's about the best you
could do on a pre-epoll era of Unix APIs. It's basically what INN and
some other Internet servers do in their core polling loop.
] [Thread Prev