Fwd: [atomic-devel] Proposing a new way to deliver Atomic systems: rpm-ostree jigdo ♲📦

[Colin Walters <walters verbum org>]

FYI, this has been my project for the last few weeks and so far it's going
really well.  If we end up going this direction for the Project Atomic/Fedora/CentOS
ecosystem realistically it will mean less of my time goes into libostree-HTTP
pull code (which unfortunately is some of the most complex), but on the
other hand I think it has the opportunity to get ostree-style deployments
onto a lot more systems.

Also, I think this work strongly showcases the *flexibility* of libostree
as a shared library, demonstrating how we can transport OSTree data
inside new/old/other formats.  This is all really only possible because
we didn't make the "checksumming compressed content" mistake
that Docker/OCI did for example.

----- Original message -----
From: Colin Walters <walters verbum org>
To: "atomic-devel" <atomic-devel projectatomic io>
Subject: [atomic-devel] Proposing a new way to deliver Atomic systems: rpm-ostree jigdo ♲📦
Date: Wed, 06 Dec 2017 13:11:11 -0500

I've been working on Project Atomic for several years now; first post:

https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2014-April/msg00000.html

(And the rpm-ostree/ostree projects predate that; rpm-ostree's "gitbirthday" is
 coming up at Sat Dec 21 19:41:30 2013 -0500)

This entire time we've dealt with 3 binary formats: RPM, Docker (now OCI) and OSTree.
The tension is...powerful.   It leaks through into how people manage systems; issues
like "how do I mirror this content" are serious blockers for lots of organizations
that don't want their systems Intenet connected.  Obviously I've thought about this a lot,
and I now have a concrete proposal now to try to get that closer to two formats.

Basically, we're reviving an old idea for the modern age of images; I'm calling
it "rpm-ostree jigdo ♲📦" (emoji are for "recycle package"):

https://github.com/projectatomic/rpm-ostree/issues/1081

I won't repaste it all here - just the header, which follows.  Feel free though
to follow up here - email works well for this sort of thing.  The high level
status is: things are moving quickly, and the next step is to start fleshing out
the client side.  So it's at the point where I want concrete feedback.

Upstream issue follows below:

Fetching content via both ostree and libdnf ends up mixing the tradeoffs of both, requires release 
engineering to manage both, and makes it harder to mirror content. Not to mention the fact that there's the 
whole OCI/Docker thing which also works in a completely different way, and admins need to manage/mirror that 
too.

Now while the "obvious" thing would be to try to align with OCI in some way, the complete lack of wire deltas 
there is very problematic for uses outside of server clusters, and given that we already have lots of 
extensive linkage to RPM via libdnf, it makes the most sense to move that direction.

Hence, let's experiment with doing ostree-in-RPM ... [ read more at 
https://github.com/projectatomic/rpm-ostree/issues/1081 ]