Re: Outgoing message truncated at dot wrapped on single line

[Brian Stafford <brian stafford uklinux net>]

On 23/10/11 19:35:33, Pawel Salek wrote:
> On 10/23/2011 07:39:12 PM, Carlos Franke wrote:
> > Balsa's debug mode does not reveil anything suspicious to me in  
> > stdout while sending a to-be-truncated email. As the ouput contains 
> 
> > some strings that look like obfuscated or encrypted passwords 
> (after
>  
> > "C: AUTH LOGIN"), I am not sure if it is safe to share it. Please  
> > point me to any reliable information source telling me that it is,  
> > and I will happily send it to you.
> 
> Auth login data is not safe to share.
> 
> > "strace -olog -ff" yields a lot of log files, only one of which  
> > contains the string "EHLO", but  n o t  the the mail content. I
> found  
> > a small portions of the mail in another log file:
> > write(12, "\nkann ich nat=C3=BCrlich versuch"..., 86) = 86
> > and
> > read(11, "\nkann ich nat=C3=BCrlich versuch"..., 86) = 86
> > What else should I look for in these files?
> 
> Yes, you are interested in the file containing EHLO. Is the session  
> encrypted? (Look for starttls in that file). Running strace -s8192
> will  
> print long string, but only if the session is unencrypted. If it is,  
> you will see only encrypted stuff. In the session is encrypted, it 
> may
>  
> be simplest to try the test program coming with libestmp.
> 
> I did following:
> 
> ./configure --with-openssl --prefix=$(HOME)/leroot
> make
> make install
> cd examples/
> PATH=$PATH:$(HOME)/leroot/bin make
> (got 'missing zError' - I had to modify Makefile to add -lz to 
> linking
>  
> line)
> ./mail-file-so -m -h SMTP_SERVER -t test-mail TARGET EXAMPLE COM
> 
> Check, if the test mail gets delivered as expected. If it is, try 
> your
>  
> test message.
> 
> You can run ./mail-file-so under strace, too.
> 
> > Eventually, the faked SMTP session with lavabit.com went fine, i. 
> e.
>  
> > the two dots were not interpreted as the end of the mail. Here is
> the  
> > complete session (for better readability, the input lines are  
> > indented):
> > $  telnet lavabit.com 25
> > [cut]
> 
> This means that a sanity test passes - which is good. It does not  
> answer the original question - it is possible Lavabit SMTP server
> makes  
> invalid assumptions about stream buffering, etc.
> 
> > 
> > Also, this session successfully sent an email to me (sender:  
> > example example com) – a somewhat distorted one, though: the Date,  
> > From, Subject and To header lines are repeated at the end of the
> body.
> 
> Not good, this should not happen.

Perhaps the following background information will help.

Headers repeating at the end of the body is indicative of unix line 
endings instead of cr-lf in the message supplied to libesmtp, so it is 
worth checking this.

libesmtp is stateful in that it process headers until it sees the 
terminating line containing only cr-lf.  It then switches to processing 
the body.  If the remote server supports BDAT (CHUNKING) no processing 
is done on the body at all, otherwise it is read a line at a time and 
dot stuffing is performed.  With unix line endings libesmtp is fooled 
into thinking it has not seen the message body.  I don't regard this as 
a bug since it is a programming error; feed libesmtp correctly and 
there is no problem.  mail-file-so has an option (--crlf) to tell it 
which style of line ending the message contains.

Regardless of this, outgoing network packets are filled as far as 
possible, usually up to about 4Kb but this buffering is done without 
regard to line endings, i.e. a line may be split across writes.  This 
should not be a problem as network applications using the TCP layer 
may not assume packet boundaries are preserved (use UDP if this 
matters).  Buffering like this is very important for performance on 
high latency links.  It also forms part of the PIPELINING support in 
libesmtp.

When using telnet to hand-craft an SMTP session, however, each network 
packet will terminate with a cr-lf.  This is quite different to 
libesmtp's behaviour.

libesmtp is nearly 10 years old; I'd be surprised if it has a dot 
stuffing bug that has only come to light now but you never know.

Brian