[ostree] doc: Add a section about how atomic upgrades work

From: Colin Walters <walters src gnome org>
To: commits-list gnome org
Cc:
Subject: [ostree] doc: Add a section about how atomic upgrades work
Date: Fri, 23 Aug 2013 17:22:11 +0000 (UTC)
commit 2b6d7d8d933144c4104b3c06abd445fc48cf197f
Author: Colin Walters <walters verbum org>
Date:   Thu Aug 22 19:14:23 2013 -0400

    doc: Add a section about how atomic upgrades work
    
    Migrating some content from
    https://live.gnome.org/OSTree/DeploymentModel2

 doc/Makefile.am         |    1 +
 doc/atomic-upgrades.xml |  201 +++++++++++++++++++++++++++++++++++++++++++++++
 doc/deployment.xml      |   11 +++
 doc/ostree-docs.xml     |    1 +
 4 files changed, 214 insertions(+), 0 deletions(-)
---
diff --git a/doc/Makefile.am b/doc/Makefile.am
index e67b650..2748a70 100644
--- a/doc/Makefile.am
+++ b/doc/Makefile.am
@@ -77,6 +77,7 @@ content_files= \
        overview.xml \
        repo.xml \
        deployment.xml \
+       atomic-upgrades.xml \
        $(NULL)
 
 # SGML files where gtk-doc abbrevations (#GtkWidget) are expanded
diff --git a/doc/atomic-upgrades.xml b/doc/atomic-upgrades.xml
new file mode 100644
index 0000000..9fb2030
--- /dev/null
+++ b/doc/atomic-upgrades.xml
@@ -0,0 +1,201 @@
+<?xml version="1.0"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
+"http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd"; [
+<!ENTITY version SYSTEM "../version.xml">
+]>
+<part id="atomic-upgrades">
+  <title>Atomic Upgrades</title>
+  <chapter id="upgrades-intro">
+    <title>You can turn off the power anytime you want...</title>
+    <para>
+      At the time of this writing, it's common for released versions
+      of mainstream operating systems (the Playstation OS, Microsoft
+      Windows, Debian GNU/Linux, and Red Hat Enterprise Linux) to have
+      non-atomic upgrades; that is, the good ones pop up a screen that
+      says "Please do not turn off the power".  The bad ones don't
+      even tell you that it's unsafe.  The <emphasis>really</emphasis>
+      bad ones actually mutate your running filesystem without taking
+      precautions to prevent corrupting running processes.
+    </para>
+
+    <para>
+      In contrast, OSTree is designed to implement fully atomic and
+      safe upgrades; more generally, atomic transitions between lists
+      of bootable deployments.  If the system crashes or you pull the
+      power, you will have either the old system, or the new one.
+    </para>
+  </chapter>
+
+  <chapter id="simple-http">
+    <title>Simple upgrades via HTTP</title>
+    <para>
+      First, the most basic model OSTree supports is one where it
+      replicates pre-generated filesystem trees from a server over
+      HTTP, tracking exactly one ref, which is stored in the <filename
+      class='extension'>.origin</filename> file for the deployment.
+      The command <command>ostree admin upgrade</command> implements
+      this.
+    </para>
+
+    <para>
+      To begin a simple upgrade, OSTree fetches the contents of the
+      ref from the remote server.  Suppose we're tracking a ref named
+      <literal>gnome-ostree/buildmaster/x86_64-runtime</literal>.
+      OSTree fetches the URL
+      
<literal>http://<replaceable>example.com</replaceable>/repo/refs/gnome-ostree/buildmaster/x86_64-runtime</literal>,
+      which contains a SHA256 checksum.  This determines the tree to
+      deploy, and <filename class='directory'>/etc</filename> will be
+      merged from currently booted tree.
+    </para>
+
+    <para>
+      If we do not have this commit, then, then we perform a pull
+      process.  At present (without static deltas), this involves
+      quite simply just fetching each individual object that we do not
+      have, asynchronously.  Put in other words, we only download
+      changed files (zlib-compressed).  Each object has its checksum
+      validated and is stored in <filename
+      class='directory'>/ostree/repo/objects/</filename>.
+    </para>
+
+    <para>
+      Once the pull is complete, we have all the objects locally
+      we need to perform a deployment.
+    </para>
+  </chapter>
+
+  <chapter id="package-manager">
+    <title>Upgrades via external tools (e.g. package managers)</title>
+
+    <para>
+      As mentioned in the introduction, OSTree is also designed to
+      allow a model where filesystem trees are computed on the client.
+      It is completely agnostic as to how those trees are generated;
+      they could be computed with traditional packages, packages with
+      post-deployment scripts on top, or built by developers directly
+      from revision control locally, etc.
+    </para>
+      
+    <para>
+      At a practical level, most package managers today
+      (<command>dpkg</command> and <command>rpm</command>) operate
+      "live" on the currently booted filesystem.  The way they could
+      work with OSTree is instead to take the list of installed
+      packages in the currently booted tree, and compute a new
+      filesystem from that.
+    </para>
+
+    <para>
+      The most basic implementation of this would be something like
+      taking the result of <command>rpm -qa</command>, and doing
+      <command>yum --installroot=/var/tmp/newroot install
+      <replaceable>package1</replaceable>
+      <replaceable>package2</replaceable> ...</command>.  Then,
+      <command>ostree commit -b osname/localtree
+      --tree=dir=/var/tmp/newroot</command>.  This would checksum all
+      of the input files and store them in local <filename
+      class='directory'>/ostree/repo</filename> repository, creating
+      a new commit.
+    </para>
+
+    <para>
+      Now, we can move on to deployment.
+    </para>
+  </chapter>
+
+  <chapter id="deployment-dir">
+    <title>Assembling a new deployment directory</title>
+    <para>
+      Given a commit to deploy, OSTree first allocates a directory for
+      it.  This is of the form <filename
+      
class='directory'>/boot/loader/entries/ostree-<replaceable>osname</replaceable>-<replaceable>checksum</replaceable>.<replaceable>serial</replaceable>.conf</filename>.
+      The <replaceable>serial</replaceable> is normally 0, but if a
+      given commit is deployed more than once, it will be incremented.
+      This is supported because the previous deployment may have
+      configuration in <filename class='directory'>/etc</filename>
+      that we do not want to use or overwrite.
+    </para>
+    
+    <para>
+      Now that we have a deployment directory, a 3-way merge is
+      performed between the (by default) currently booted deployment's
+      <filename class='directory'>/etc</filename>, its default
+      configuration, and the new deployment (based on its <filename
+      class='directory'>/usr/etc</filename>).
+    </para>
+  </chapter>
+
+  <chapter id="swapping-boot">
+    <title>Atomically swapping boot configuration</title>
+    <para>
+      At this point, a new deployment directory has been created as a
+      hardlink farm; the running system is untouched, and the
+      bootloader configuration is untouched.  We want to add this deployment
+      to the "deployment list".
+    </para>
+
+    <para>
+      To support a more general case, OSTree supports atomic
+      transitioning between arbitrary sets of deployments, with the
+      restriction that the currently booted deployment must always be
+      in the new set.  In the normal case, we have exactly one
+      deployment, which is the booted one, and we want to add the new
+      deployment to the list.  A more complex command might allow
+      creating 100 deployments as part of one atomic transaction, so
+      that one can set up an automated system to bisect across them.
+    </para>
+
+    <simplesect id="bootversion">
+      <title>The bootversion</title>
+      <para>
+       OSTree allows swapping between boot configurations by
+       implementing the "swapped directory pattern" in <filename
+       class='directory'>/boot</filename>.  This means it is a
+       symbolic link to one of two directories <filename
+       class='directory'>/ostree/boot.<replaceable>[0|1]</replaceable></filename>.
+       To swap the contents atomically, if the current version is
+       <literal>0</literal>, we create <filename
+       class='directory'>/ostree/boot.1</filename>, populate it with
+       the new contents, then atomically swap the symbolic link.  Finally,
+       the old contents can be garbage collected at any point.
+      </para>
+    </simplesect>
+
+    <simplesect id="ostree-bootversion">
+      <title>The /ostree/boot directory</title>
+      <para>
+       However, we want to optimize for the case where we the set of
+       kernel/initramfs pairs is the same between both the old and
+       new deployment lists.  This happens when doing an upgrade that
+       does not include the kernel; think of a simple translation
+       update.  OSTree optimizes for this case because on some
+       systems <filename class='directory'>/boot</filename> may be on
+       a separate medium such as flash storage not optimized for
+       significant amounts of write traffic.
+      </para>
+
+      <para>
+       To implement this, OSTree also maintains the directory
+       <filename
+       class='directory'>/ostree/boot.<replaceable>bootversion</replaceable></filename>,
+       which is a set of symbolic links to the deployment
+       directories.  The <replaceable>bootversion</replaceable> here
+       must match the version of <filename
+       class='directory'>/boot</filename>.  However, in order to
+       allow atomic transitions of <emphasis>this</emphasis>
+       directory, this is also a swapped directory, so just like
+       <filename class='directory'>/boot</filename>, it has a version
+       of <literal>0</literal> or <literal>1</literal> appended.
+      </para>
+
+      <para>
+       Each bootloader entry has a special <literal>ostree=</literal>
+       argument which refers to one of these symbolic links.  This is
+       parsed at runtime in the initramfs.
+      </para>
+
+    </simplesect>
+
+  </chapter>
+
+</part>
diff --git a/doc/deployment.xml b/doc/deployment.xml
index 69216e7..489850a 100644
--- a/doc/deployment.xml
+++ b/doc/deployment.xml
@@ -111,6 +111,17 @@
       class='directory'>/usr</filename>, but this is not a hard
       requirement.
     </para>
+
+    <para>
+      Finally, a deployment may have a <filename
+      class='extension'>.origin</filename> file, stored next to its
+      directory.  This file tells <command>ostree admin
+      upgrade</command> how to upgrade it.  At the moment, OSTree only
+      supports upgrading a single refspec.  However, in the future
+      OSTree may support a syntax for composing layers of trees, for
+      example.
+    </para>
+
   </chapter>
   
   <chapter id="managing-boot">
diff --git a/doc/ostree-docs.xml b/doc/ostree-docs.xml
index 46b4cd0..cf953c1 100644
--- a/doc/ostree-docs.xml
+++ b/doc/ostree-docs.xml
@@ -14,6 +14,7 @@
        <xi:include href="overview.xml"/>
        <xi:include href="repo.xml"/>
        <xi:include href="deployment.xml"/>
+       <xi:include href="atomic-upgrades.xml"/>
 
        <chapter xml:id="reference">
                <title>API Reference</title>
[Date Prev][Date Next] [Thread Prev][Thread Next] [Thread Index] [Date Index] [Author Index]