Re: Patch to work around flaky pop3 server behaviour

[Brian Stafford <brian stafford uklinux net>]

On 2001.06.09 18:35:25 +0100 Carlos Morgado wrote:
> On Sat, Jun 09, 2001 at 06:17:29PM +0100, Brian Stafford wrote:

> > Incidentally there should be no problem reading more than one character
> > at a time even with blocking sockets.  read() preserves packet
> boundaries
> > when reading from a socket.  Since the last characters in the packet
> 
> no it doesn't. it usually does, but you can get short reads that won't
> turn
> up an error. revc and especially read don't garantee packet boundries.
> you
> can play with RECV_LOW_WATERMARK but even that doesn't garantee anything.

Fair enough.  I was actually referring to my understanding that read()
will not read beyond a packet boundary in a single read().

> > during command response exchanges will be the \r\n, there is no need
> > to read input a character at a time scanning for the line termination.
> 
> now you're making assumptions about silly stuff like line MTU and server
> side smartness. this is will break (it always does) eventually and 
> noone will be able to figure out why without throwing tcpdump at it.

We're dealing with the specific case of POP-3.  All POP exchanges
eventually terminate with the \r\n and unless pipelining, the transmitter
turns the line round and reads.  So when reading buffers, if the last
characters in the buffer are not \r\n, read() should be called again.

> > This is a property of protocols such as POP, IMAP, SMTP etc.
> > 
> don't forget there's tcp and ip and ppp and hdlc and lots of assorted
> nastiness underneath that

Its true that this stuff matters but it's not relevant for POP.  In
practice all that happens is that read returns less data than was sent
by the transmitter and since the \r\n is not present, just call read()
again.

What I'm saying is that we can take advantage of the characteristics
of the protocol we're using and avoid single character reads.  What
I'm not saying is that this is a general solution.

> > In fact, unless the POP server supports PIPELINING and the client uses
> > it, there is no possibility of a blocking read() trying to read beyond
> > the end of the line.  Reading a socket for lock step protocol exchanges
> 
> unless the server says so.

In this case, the server advertises PIPELINING in response to the CAPA
command, and the client is coded to cope with PIPELINING.  Unless the
client batches multiple commands, PIPELINING POP servers behave identically
to non-PIPELINING servers.

> 
> > is not the same problem as reading a tty which really must be done
> > character by character (unless using half-delay mode which sockets
> don't
> > have).
> > 
> 
> half-delay ? you mean the nagle(humm .. i think that's it) tcp buffering
> protocol ?

No, I'm referring to the half-delay mode in the termios interface to ttys.
The tty driver can be asked to blcok until a minimum number of characters
have been read (usually 1) and then delay for a short time (usally 100ms)
in case more characters arrive.  When the timeout expires read() returns
with at least the minumum number of characters in the buffer.  The
combination of minimum + time delay is sometimes called half-delay mode.

> > While on the topic of libESMTP, it might be worth considering the use
> > of the socket buffering in siobuf.c for reading and writing in Balsa's
> > POP client.  (Perhaps I should add a POP client to libESMTP :)
> > 
> 
> actually, balsa should use glib io channel stuff. it's all neatly

OK - my thinking is guided by the objective of avoiding glib dependencies
in libESMTP.  I would agree with that approach in a glib program.

> integrated
> into gtk main loop and saves us the trouble of doing select/poll
> ourselfs.
> otoh, it makes everything socket related be handled by callbacks which is
> an
> heap of extra work itself

This is the down side of callbacks.

> > A few other points:
> > 
> > > i like select/poll with timeouts better than signals and much better
> than
> > > fnctl tricks
> > 
> > Firstly, fcntl() is not a trick.  This is the standard way of getting
> > a non blocking socket, or indeed any type of non blocking file
> 
> i meant pingponging the descriptor from block to nonblocking. that's a
> dirty
> and costly trick. (read the original patch)

Oh sorry, I misunderstood.  I agree with you here actually.  The socket
should either be blocking or non-blocking.  Since poll() or select()
can do the waiting the socket should never need to be switched back
and forth from non-blocking mode.

> > Secondly, poll() should be used in preference to select() in Posix
> code.
> > Select(), IMO, is an appalling piece of design.
> > 
> works well for small sets and exists everywhere. i see no very compeling
> reason to use poll unless you're writting a ftp server.

Yes, but its only OK for small sets.  Its still dreadful design.
The compelling reason for poll() IMO is that its Posix and select()
isn't.  As I understand it, in recent versions of Linux, poll() is the
primary implementation and select() is layered over it, so I assume
that poll() is better on Linux at least.  I have also used only poll()
in libESMTP to date and received no complaints that select() is
required.  I assume that poll() being part of Posix is assuring its
wide availability.

Regards
Brian