deltas, history pruning vs GPG

[Colin Walters <walters verbum org>]

An important and large part of the OSTree problem domain boils down to getting files from A to B across the 
Internet securely.

Obviously, many software systems out there approach this, but if you add in a few characteristics:
 - Servable over plain static HTTP
 - Content is managed as versioned immutable snapshots
 - Unidirectional flow
 - Focus on executable code

That eliminates things like Dropbox, rsync, git, etc. in the sense
that while some some components of those projects might be
useful, they're sufficiently different that implementation choices
would vary significantly.

The general class of project then falls into:
 - Package systems
 - Language/App updaters
 - Basic images (ChromeOS)
 - Complex images (Docker)

This is also the same problem domain that is a focus of:

http://theupdateframework.com/

Which I've been looking at again.

Basically, what I'm wrestling with now is whether or not deltas are GPG signed
individually.

At a high level, with respect to security, there are two high level approaches that I consider "strong":
- TLS with pinned CA cert(s)  - This model is used by yum with Red Hat Enterprise Linux,
  and OSTree supports this as well via the tls-ca-path option.  Particularly
  good if you also ensure a reasonable minimum cipher suite (not yet implemented with
  OSTree/gnutls).
- GPG - Also implemented by OSTree, commits can be GPG signed.

The TLS model provides strong integrity and confidentiality independent of the repository,
but makes it harder to mirror content, and does not allow offline verification.  (Though
certainly we could record on-disk that the content was fetched from a cert signed
by a particular CA)

The GPG model doesn't provide confidentiality itself, but allows mirroring and offline
verification, but requires management in the repository.  I'd personally really like to
have good GPG support for these reasons.  Even if pinned TLS is in use, GPG
provides another very strong layer of security.

So, the current deltas design has them being signed separately from the commits.
This is distinct from e.g. deltarpm, which goes
to some lengths to reconstruct the original binary RPM in order to enable GPG verification.

(This after-the-fact implementation also impacts the deltarpm disk/CPU consumption)

Additionally, we don't presently support signing the summary flie, but doing so
would prevent a class of attack raised by Florian here:
https://mail.gnome.org/archives/ostree-list/2014-February/msg00010.html

> The name of the repository head is not covered by the GnuPG signature. As a result, a man-in-the-middle 
> attack can cause an
> image to upgrade to a totally unrelated head from the same repository (because it is signed with the same 
> trusted key).

Finally, another concern that was just raised on list is around trimming repository
history.  If we support that, it would necessarily entail re-signing all of the newly
generated commit history.

These types of concerns are leading me to circle back to what Debian/apt
does by default, which is sign the Release file (equivalent of OSTree's "summary").
See https://wiki.debian.org/SecureApt#Signed_Release_files

yum also supports this, but notably it's not used by Fedora, only individual
packages are signed.

What then becomes an important topic is mapping this to the OSTree history.
As far as I know, Debian just regenerates that file, and doesn't maintain any
history.  (This is where http://snapshot.debian.org/ comes in)

However, I really want that built into OSTree itself, because it's a fundamental
part of enabling continuous delivery.  See 
http://dustymabe.com/2015/01/31/crisis-averted-im-using-atomic-host/

So an idea right now is that the summary file is versioned and signed, and
the summary file contains sha256 of the commits and deltas.

It could look like this:

 repo/summary  (current)
 repo/summary.201501
 repo/summary.201412
 repo/summary/201411

(Obviously with a configurable rotation schedule, but monthly would
 probably be a good default)

The current summary could also contain something like a bloom
filter for which checksums are in previous summaries, to avoid
having to do a reverse chronological search.

This also better supports rebasing history, as the commit checksum
would stay around in the summary file, permanently marked as
"known trusted", but the content could go away.