Re: [Evolution] gpg help

[Not Zed <notzed ximian com>]

On Tue, 2002-06-04 at 17:03, Adrian 'Dagurashibanipal' von Bidder wrote:
> On Mon, 2002-06-03 at 10:40, Not Zed wrote:
> > On Mon, 2002-06-03 at 18:21, Adrian 'Dagurashibanipal' von Bidder wrote:
> [...]
>
> > > signatures not validating half of the time makes pgp/gpg look very
> > > unreliable, which it isn't in my experience (it works basically
> > > everywhere except in evolution...)
> >
> > Well, inline pgp suddenly stipulates that a text/plain part must be
> > treated the same way as an 8 bit or binary part.  As completely opaque
> > data.  We store data decoded in memory, and so this breaks this
> > requirement.  And we're not about to change it.
>
> why not make signature verifying a part of the decoding process - if
> you're insisting that you want to store decoded data that would probably
> be the only way.
>
> But I agree that inline pgp is quite ugly, because you don't know it's
> there before having scanned the text.

Its abstracted away.  The mime decoder only parses the mime level.  It
could also check for inline crap like postscript and uuencoded data, but
it would have to be after a transfer-decoding pass, which would be very
difficult.

Also, i wouldn't trust a signature that wasn't done on the spot.  Some
meta-data to indicate it was valid doesn't sound too hot to me.  Anyway,
yeah, we had (well i had) already thought of that and dismissed it as
not practical for camel or ideal.

Anyway, this isn't to say that it wont be possible in the future, only
i'm not in a rush to do it, and its too big a job to get a patch for,
unless someone is willing to put a great deal of effort into it (and
unfortunately, often, big patches are more work than they save).  I did
start to work on changing the way camel did things, so that the mailer
would have to do more of the work.  There are actually some other
reasons why this might be a good idea, mainly for performance (although
that isn't really a problem with camel, at least not in this context).

I found that it would be would be a lot of work to reorganise where and
when various operations are done in camel/the mailer.  I even started
writing a big long proposal/discussion of the changes required, but i
settled for the much simpler task of just getting multipart/signed to
work on its own - at least in that case we have a specific mime-type and
a specific mime-related rfc on how it should work, and it can be done
transparently as part of our normal decoding processes.  And it is
needed for s/mime too.  Probably 1/10th of the work required to
implement it, I was losing enough sleep as it was!

So, it was a tradeoff.  openpgp lost unfortunately (at least for now, I
*had* intended to put in place code that would allow it to work).  I
would personally rather see s/mime working.  Who knows, maybe one wet
weekend i'll get jacked up on coffee and do it, but i wouldn't hold my
breath if I were you :)

> > > (It seems building from cvs is a bit too complicated, as some of the
> [...]
>
> Ok, thanks (Rodrigo, you too) for clarifying this. Which branches am I
> to take?`evo HEAD goes together with gal and gtkhtml HEAD, too?

Yep.

> > > > The multipart/signed rfc's are broken, they break valid assumptions you
> > > [...]
> > > > if for example any mailer blows apart mime parts and stores them
> > > > decoded, which imho is a perfectly valid thing to want to do).  But
> > >
> > > Hmmm. I'd agree that this may be a valid thing to do in the local cache.
> > > But I strongly feel that the mailer changing the msg body stored in the
> > > mail spool (or imap dir) without being told so is broken.
> >
> > So, how do you then move a message to a system that supports different
> > character sets?  Should you just throw it away?  No, you translate the
> > character sets?  What about if a mail is sent in 7 bit format, but your
> > transport supports an 8 bit or binary format?  Or visa versa?  It is
>
> A MTA should NEVER eat/change anything in the body of the mail. An MTA
> shouldn't even need to know about MIME (that's why MIME was designed in
> the first place). MIME allows anything to be encoded 7bit clean, so
> there should not be any MTA barfing on any MIME mail. 
>
> Today, most MTAs are 8bit clean, so some MUA send 8 bit mail, but I find
> this to be a bit risky, as there are still some 7bit MTAs out there
> (and, IIRC exim for example does not advertise it's 8bit capabilities
> per default). So 8 bit mail --> 7 bit MTA is the only problem I can see.
> Bouncing the message in this case is not really acceptable, but I don't
> think breaking signed mails for this case can be avoided. But - as I
> said - I feel MIME mail should always be 7bit clean.

> Decoding - and transforming charsets - is something that belongs at the
> very end where the mail is displayed.

Well rfc821 says it can be done, but only if it is reversible if sent
onwards.

Now the problem with the way the multipart/mime stuff has been done is
this: even if you get a bit-for-bit decoded copy of the original
content, which quoted-printable can do pretty reliably these days, and
base64 can do very reliably - it will still not necessarily verify. 
That is because there are ambiguities in the way you can decide to qp or
base64 the data.  Where you line-break, etc.  I dont know *why* they
didn't say 'undo the transfer-encoding and then pass it to us and we'll
verify', but they said instead 'do the transfer encoding', then
sign/verify on that.  They could've even just said 'with base64, you
strip out all invalid chars, and line-break every 72 characters', etc. 
But the 'canonical encoding' isn't canonical at all, you can do it
multiple ways, so it basically rules out just about anything any client
could do to optimise storage, etc.  Forget database backends (unless,
ugh, you store everything in raw format!).


So, I guess the stuff I said below about the transport was basically
completely wrong, duh, i need a holiday.  Rfc821 covers this to some
extent.

However, rfc2049 basically says its unreliable to assume that things
like that wont happen in the real world, so no code should rely upon it
... 

Particularly, see section 3, note (1) - 

    (1)   Under some circumstances the encoding used for data may
          change as part of normal gateway or user agent
          operation.  In particular, conversion from base64 to
          quoted-printable and vice versa may be necessary.  This
          may result in the confusion of CRLF sequences with line
          breaks in text bodies.  As such, the persistence of
          CRLF as something other than a line break must not be
          relied on.

Yes, it later goes on to say that things shouldn't do this ... but they
do.

The multipart/signed design should have taken this kind of
comment/advice into account (it instead relies absolutely on the precise
position and exact number of crlf sequences in the encoded text (or
encoded binary for that matter) for example, not the viewed text).

> (Of course, how MTAs and MUAs should deal with broken MIME and non-MIME
> messages is another question alltogether... and unfortunately one that
> comes up frequently)

Yes, unfortunately there are a *lot* of broken mailers out there. 
Sending and receiving ...