[ostree] README: Just link to wiki, move most of it to README-historical.md



commit 35c1fe324fca741e2b248202c775392279286d0a
Author: Colin Walters <walters verbum org>
Date:   Mon Jan 20 18:00:09 2014 -0500

    README: Just link to wiki, move most of it to README-historical.md

 README-historical.md |  451 +++++++++++++++++++++++++++++++++++++++++++++++++
 README.md            |  457 +-------------------------------------------------
 2 files changed, 459 insertions(+), 449 deletions(-)
---
diff --git a/README-historical.md b/README-historical.md
new file mode 100644
index 0000000..732f8bd
--- /dev/null
+++ b/README-historical.md
@@ -0,0 +1,451 @@
+This file is outdated, but some of the text here is still useful for
+historical context.  I'm preserving it (explicitly still in the tree)
+for posterity.
+
+OSTree
+======
+
+Problem statement
+-----------------
+
+Hacking on the core operating system is painful - this includes most
+of GNOME from upower and NetworkManager up to gnome-shell.  I want a
+system that matches these requirements:
+
+1. Does not disturb your existing OS
+2. Is not terribly slow to use
+3. Shares your $HOME - you have your data
+4. Allows easy rollback
+5. Ideally allows access to existing apps
+
+Comparison with existing tools
+------------------------------
+
+ - Virtualization
+
+    Fails on points 2) and 3).  Actually qemu-kvm can be pretty fast,
+    but in a lot of cases there is no substitute for actually booting
+    on bare metal; GNOME 3 really needs some hardware GPU
+    acceleration.
+
+ - Rebuilding distribution packages
+
+    Fails on points 1) and 4).  Is also just very annoying: dpkg/rpm
+    both want tarballs, which you don't have since you're working from
+    git.  The suggested "mock/pbuilder" type chroot builds are *slow*.
+    And even with non-chroot builds there is lots of pointless build
+    wrapping going on.  Both dpkg and rpm also are poor at helping you
+    revert back to the original system.
+
+    All of this can be scripted to be less bad of course - and I have
+    worked on such scripts.  But fundamentally you're still fighting
+    the system, and if you're hacking on a lowlevel library like say
+    glib, you can easily get yourself to the point where you need a
+    recovery CD - at that point your edit/compile/debug cycle is just
+    too long.
+
+ - "sudo make install"
+
+    Now your system is in an undefined state.  You can use e.g. rpm
+    -qV to try to find out what you overwrote, but neither dpkg nor
+    rpm will help clean up any files left over that aren't shipped by
+    the old package.  
+
+    This is most realistic option for people hacking on system
+    components currently, but ostree will be better.
+
+ - LXC / containers
+
+   Fails on 3), and 4) is questionable.  Also shares the annoying part
+   of rebuilding distribution packages. LXC is focused on running
+   multiple server systems at the *same time*, which isn't what we
+   want (at least, not right now), and honestly even trying to support
+   that for a graphical desktop would be a lot of tricky work, for
+   example getting two GDM instances not to fight over VT
+   allocations. But some bits of the technology may make sense to use.
+
+ - jhbuild + distribution packages
+
+    The state of the art in GNOME - but can only build non-root things -
+    this means you can't build NetworkManager, and thus are permanently
+    stuck on whatever the distro provides.
+
+Who is ostree for?
+------------------
+
+First - operating system developers and testers.  I specifically keep
+a few people in mind - Dan Williams and Eric Anholt, as well as myself
+obviously.  For Eric Anholt, a key use case for him is being able to
+try out the latest gnome-shell, and combine it with his work on Mesa,
+and see how it works/performs - while retaining the ability to roll
+back if one or both breaks.
+
+The rollback concept is absolutely key for shipping anything to
+enthusiasts or knowledable testers.  With a system like this, a tester
+can easily perform a local rollback - something just not well
+supported by dpkg/rpm.  (What about Conary?  See below.)
+
+Also, distributing operating system trees (instead of packages) gives
+us a sane place to perform automated QA **before** we ship it to
+testers.  We should never be wasting these people's time.
+
+Even better, this system would allow testers to *bisect* across
+operating system builds, and do so very efficiently.
+
+The core idea - chroots
+-----------------------
+
+chroots are the original lightweight "virtualization".  Let's use
+them.  So basically, you install a mainstream distribution (say
+Debian).  It has a root filesystem like this:
+
+               /usr
+               /etc
+               /home
+               ...
+
+Now, what we can do is have a system that installs chroots as a subdirectory
+of the root, like:
+
+               /ostree/gnomeos-3.0-opt-393a4555/{usr,etc,sbin,...}
+               /ostree/gnomeos-3.2-opt-7e9788a2/{usr,etc,sbin,...}
+
+These live in the same root filesystem as your regular distribution
+(Note though, the root partition should be reasonably sized, or
+hopefully you've used just one big partition).
+
+You should be able to boot into one of these roots.  Since ostree
+lives inside a distro created partition, a tricky part here is that we
+need to know how to interact with the installed distribution's grub.
+This is an annoying but tractable problem.
+
+First, we install a kernel+initramfs alongside the distribution's.
+Then, we have a "trampoline" ostree-init binary which is statically
+linked, and boot the kernel with init=/ostree/ostree-init.  This then
+takes care of chrooting and running the init binary.
+
+An important note here is that we bind mount the real /home.  This
+means you have your data.  This also implies we share uid/gid, so
+/etc/passwd will have to be in sync.  Probably what we'll do is have a
+script to pull the data from the "host" OS.
+
+I've decided for now to move /var into /ostree to avoid sharing it
+with the "host" distribution, because in practice we're likely
+to hit incompatibilities.
+
+Do note however /etc lives *inside* the OSTree; it's presently
+versioned and readonly like everything else.
+
+On a pure OSTree system, the filesystem layout will look like this:
+
+               .
+               |-- boot
+               |-- home
+               |-- ostree
+               |   |-- var
+               |   |-- current -> gnomeos-3.2-opt-7e9788a2
+               |   |-- gnomeos-3.0-opt-393a4555
+               |   |   |-- etc
+               |   |   |-- lib
+               |   |   |-- mnt
+               |   |   |-- proc
+               |   |   |-- run
+               |   |   |-- sbin
+               |   |   |-- srv
+               |   |   |-- sys
+               |   |   `-- usr
+               |   `-- gnomeos-3.2-opt-7e9788a2
+               |       |-- etc
+               |       |-- lib
+               |       |-- mnt
+               |       |-- proc
+               |       |-- run
+               |       |-- sbin
+               |       |-- srv
+               |       |-- sys
+               |       `-- usr
+               |-- root
+               
+
+Making this efficient
+---------------------
+
+One of the first things you'll probably ask is "but won't that use a
+lot of disk space"?  Indeed, it will, if you just unpack a set of RPMs
+or .debs into each root.
+
+Besides chroots, there's another old Unix idea we can take advantage
+of - hard links.  These allow sharing the underlying data of a file,
+with the tradeoff that changing any one file will change all names
+that point to it.  This mutability means that we have to either:
+
+ 1. Make sure everything touching the operating system breaks hard links
+    This is probably tractable over a long period of time, but if anything
+    has a bug, then it corrupts the file effectively.
+ 2. Make the core OS read-only, with a well-defined mechanism for mutating
+    under the control of ostree.
+
+I chose 2.
+
+A userspace content-addressed versioning filesystem
+---------------------------------------------------
+
+At its very core, that's what ostree is.  Just like git.  If you
+understand git, you know it's not like other revision control systems.
+git is effectively a specialized, userspace filesystem, and that is a
+very powerful idea.
+
+At the core of git is the idea of "content-addressed objects".  For
+background on this, see <http://book.git-scm.com/7_how_git_stores_objects.html>
+
+Why not just use git?  Basically because git is designed mainly for
+source trees - it goes to effort to be sure it's compressing text for
+example, under the assumption that you have a lot of text.  Its
+handling of binaries is very generic and unoptimized.
+
+In contrast, ostree is explicitly designed for binaries, and in
+particular one type of binary - ELF executables (or it will be once we
+start using bsdiff).  
+
+Another big difference versus git is that ostree uses hard links
+between "checkouts" and the repository.  This means each checkout uses
+almost no additional space, and is *extremely* fast to check out.  We
+can do this because again each checkout is designed to be read-only.
+
+So we mentioned above there are:
+
+               /ostree/gnomeos-3.2-opt-7e9788a2
+               /ostree/gnomeos-3.2-opt-393a4555
+
+There is also a "repository" that looks like this:
+
+               /ostree/repo/objects/17/a95e8ca0ba655b09cb68d7288342588e867ee0.file
+               /ostree/repo/objects/17/68625e7ff5a8db77904c77489dc6f07d4afdba.meta
+               /ostree/repo/objects/17/cc01589dd8540d85c0f93f52b708500dbaa5a9.file
+               /ostree/repo/objects/30
+               /ostree/repo/objects/30/6359b3ca7684358a3988afd005013f13c0c533.meta
+               /ostree/repo/objects/30/8f3c03010cedd930b1db756ce659c064f0cd7f.meta
+               /ostree/repo/objects/30/8cf0fd8e63dfff6a5f00ba5a48f3b92fb52de7.file
+               /ostree/repo/objects/30/6cad7f027d69a46bb376044434bbf28d63e88d.file
+
+Each object is either metadata (like a commit or tree), or a hard link
+to a regular file.
+
+Note that also unlike git, the checksum here includes *metadata* such
+as uid, gid, permissions, and extended attributes.  (It does not include
+file access times, since those shouldn't matter for the OS)
+
+This is another important component to allowing the hardlinks.  We
+wouldn't want say all empty files to be shared necessarily.  (Though
+maybe this is wrong, and since the OS is readonly, we can make all
+files owned by root without loss of generality).
+
+However this tradeoff means that we probably need a second index by
+content, so we don't have to redownload the entire OS if permissions
+change =)
+
+Atomic upgrades, rollback
+-------------------------
+
+OSTree is designed to atomically swap operating systems - such that
+during an upgrade and reboot process, you either get the full new
+system, or the old one.  There is no "Please don't turn off your
+computer".  We do this by simply using a symbolic link like:
+
+/ostree/current -> /ostree/gnomeos-3.4-opt-e3b0c4429
+
+Where gnomeos-e3b0c4429 has the full regular filesystem tree with usr/
+etc/ directories as above.  To upgrade or rollback (there is no
+difference internally), we simply check out a new tree into
+gnomeos-b90ae4763 for example, then swap the "current" symbolic link,
+then remove the old tree.
+
+But does this mean you have to reboot for OS upgrades?  Very likely,
+yes - and this is no different from RPM/deb or whatever.  They just
+typically lie to you about it =)
+
+A typical model with RPM/deb is to unpack the new files, then use some
+IPC mechanism (SIGHUP, a control binary like /usr/sbin/apachectl) to
+signal the running process to reload.  There are multiple problems
+with this - one is that in the new state, daemon A may depend on the
+updated configuration in daemon B.  This may not be particularly
+common in default configurations, but it's highly likely that that
+some deployments will have e.g. apache talking to a local MySQL
+instance.  So you really want to do is only apply the updated
+configuration when all the files are in place; not after each RPM or
+.deb is installed.
+
+What's even harder is the massive set of race conditions that are
+possible while RPM/deb are in the process of upgrading.  Cron jobs are
+very likely to hit this.  If we want the ability to apply updates to a
+live system, we could first pause execution of non-upgrade userspace
+tasks.  This could be done via SIGSTOP for example.  Then, we can swap
+around the filesystem tree, and then finally attempt to apply updates
+via SIGHUP, and if possible, restart processes.
+
+Configuration Management
+------------------------
+
+By now if you've thought about this problem domain before, you're wondering
+about configuration management.  In other words, if the OS is read only,
+how do I edit /etc/sudoers?
+
+Well, have you ever been a system administrator on a zypper/yum
+system, done an RPM update, which then drops .rpmnew files in your
+/etc/ that you have to go and hunt for with "find" or something, and
+said to yourself, "Wow, this system is awesome!!!" ?  Right, that's
+what I thought.
+
+Configuration (and systems) management is a tricky problem, and I
+certainly don't have a magic bullet.  However, one large conceptual
+improvement I think is defaulting to "rebase" versus "merge".
+
+This means that we won't permit direct modification of /etc - instead,
+you HAVE to write a script which accomplishes your goals.  To generate
+a tree, we check out a new copy, then run your script on top.
+
+If the script fails, we can roll back the update, or drop to a shell
+if interactive.
+
+However, we also need to consider cases where the administrator
+modifies state indirectly by a program.  Think "adduser" for example.
+
+Possible approaches:
+
+ 1. Patch all of these programs to know how to write to the writable
+    location, instead of the R/O bind mount overlay.
+ 2. Move the data to /var
+
+What about "packages"?
+----------------------
+
+There are several complex and separate issues hiding in this seemingly
+simple question. 
+
+I think OSTree always makes sense to use as a core operating system
+builder and updater.  By "core" here I mean the parts that aren't
+removable.  Debian has Essential: yes, any other distribution has this
+too implicitly in the set of dependencies for their updater tool.
+
+Now, let me just say I will absolutely support using something like
+apt/yum/zypper (and consequently deb/rpm) on top of OSTree.  This
+isn't trivial, but there aren't any conceptual issues.
+
+Concretely for example, RPM or .deb might make sense as a delivery
+vehicle for third party OS extensions.  A canoncial example is the
+NVidia graphics driver.
+
+If one is using OSTree to build an *operating system*, then there has
+to be some API for applications.  And that demands its own targeted
+solution - something like an evolved glick (zeroinstall is also
+similar).
+
+Current package systems are totally broken for application deployment
+though; for example, they will remove files away from under running
+applications on update.  And we clearly need the ability to install
+and upgrade applications without rebooting the OS.
+
+Details of RPM installation
+---------------------------
+
+We should be able to install LSB rpms.  This implies providing "rpm".
+The tricky part here is since the OS itself is not assembled via RPMs,
+we need to fake up a database of "provides" as if we were.  Even
+harder would be maintaining binary compatibilty with any arbitrary
+%post scripts that may be run.
+
+What about BTRFS?  Doesn't it solve everything?
+-----------------------------------------------
+
+In short, BTRFS is not a magic bullet, but yes - it helps
+significantly.  The obvious thing to do is layer BTRFS under dpkg/rpm,
+and have a separate subvolume for /home so rollbacks don't lose your
+data.  See e.g.
+<http://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs>
+
+As a general rule an issue with the BTRFS is that it can't roll back
+just changes to things installed by RPM (i.e. what's in rpm -qal).
+
+For example, it's possible to e.g. run yum update, then edit something
+in /etc, reboot and notice things are broken, then roll back and have
+silently lost your changes to /etc.
+
+Another example is adding a new binary in /usr/local.  You could say,
+"OK, we'll use subvolumes for those!".  But then what about /var (and
+your VM images that live in /var/lib/libvirt ?)
+
+Finally, probably the biggest disadvantage of the rpm/dpkg + BTRFS
+approach is it doesn't solve the race conditions that happen when
+unpacking packages into the live system.  This problem is really
+important to me.
+
+Note though ostree can definitely take advantage of BTRFS features!
+In particular, we could use "reflink"
+<http://lwn.net/Articles/331808/> instead of hard links, and avoid
+having the object store corrupted if somehow the files are modified
+directly.
+
+Other systems
+-------------
+
+I've spent a long time thinking about this problem, and here are some
+of the other possible solutions out there I looked at, and why I
+didn't use them:
+
+ - Git: <http://git-scm.com/>
+
+    Really awesome, and the core inspiration here.  But like I mentioned
+    above, not at all designed for binaries - we can make different tradeoffs.
+
+ - bup: <https://github.com/apenwarr/bup>
+
+    bup is cool.  But it shares the negative tradeoffs with git, though it
+    does add positives of its own.  It also inspired me.
+
+ - git-annex: <http://git-annex.branchable.com/git-annex/>
+
+   Looks interesting; I think this will share the same negative tradeoffs with git
+   as far as using it for an OS goes.
+
+ - schroot: <http://www.debian-administration.org/articles/566>
+
+   Like LXC/containers, but just using a chroot.
+
+ - NixOS: <http://nixos.org/>
+
+    The NixOS people have a lot of really good ideas, and they've definitely
+    thought about the problem space.  However, their approach of checksumming
+    all inputs to a package is pretty wacky.  I don't see the point, and moreover
+    it uses gobs of disk space.
+
+ - Conary: <http://wiki.rpath.com/wiki/Conary:Updates_and_Rollbacks>
+
+   If rpm/dpkg are like CVS, Conary is closer to Subversion.  It's not
+   bad, but ostree is better than it for the exact same reasons git
+   is better than Subversion.
+
+ - BTRFS: <http://en.wikipedia.org/wiki/Btrfs>
+
+   See above.
+
+ - Solaris IPS: <http://hub.opensolaris.org/bin/view/Project+pkg/>
+
+   Rollback is ZFS level, so I think this shares the same tradeoffs as BTRFS+RPM/deb.
+   They probably have some vertical integration though which definitely helps.
+   Obviously we can't use ZFS.
+
+ - Jhbuild: <https://live.gnome.org/Jhbuild>
+
+   What we've been using in GNOME, and has the essential property of allowing you
+   to "fall back" to a stable system.  But ostree will blow it out of the water.
+
+Development
+-----------
+
+ - OSTree wiki page: <https://live.gnome.org/OSTree>
+
+ - ostbuild wiki page: <https://live.gnome.org/OSTree/Ostbuild>
+
+ - Git repository: <http://git.gnome.org/browse/ostree/>
+
+ - Deploying OSTree in the Gnome servers: <https://bugzilla.gnome.org/show_bug.cgi?id=669772>
diff --git a/README.md b/README.md
index 333ba7c..6e5d8b5 100644
--- a/README.md
+++ b/README.md
@@ -1,451 +1,10 @@
-Note, I'm slowly trying to move some of this content to
+OSTree is a tool for managing bootable, immutable, versioned
+filesystem trees. While it takes over some of the roles of tradtional
+"package managers" like dpkg and rpm, it is not a package system; nor
+is it a tool for managing full disk images. Instead, it sits between
+those levels, offering a blend of the advantages (and disadvantages)
+of both.
 
-https://live.gnome.org/OSTree
+For more information, see:
 
-OSTree
-======
-
-Problem statement
------------------
-
-Hacking on the core operating system is painful - this includes most
-of GNOME from upower and NetworkManager up to gnome-shell.  I want a
-system that matches these requirements:
-
-1. Does not disturb your existing OS
-2. Is not terribly slow to use
-3. Shares your $HOME - you have your data
-4. Allows easy rollback
-5. Ideally allows access to existing apps
-
-Comparison with existing tools
-------------------------------
-
- - Virtualization
-
-    Fails on points 2) and 3).  Actually qemu-kvm can be pretty fast,
-    but in a lot of cases there is no substitute for actually booting
-    on bare metal; GNOME 3 really needs some hardware GPU
-    acceleration.
-
- - Rebuilding distribution packages
-
-    Fails on points 1) and 4).  Is also just very annoying: dpkg/rpm
-    both want tarballs, which you don't have since you're working from
-    git.  The suggested "mock/pbuilder" type chroot builds are *slow*.
-    And even with non-chroot builds there is lots of pointless build
-    wrapping going on.  Both dpkg and rpm also are poor at helping you
-    revert back to the original system.
-
-    All of this can be scripted to be less bad of course - and I have
-    worked on such scripts.  But fundamentally you're still fighting
-    the system, and if you're hacking on a lowlevel library like say
-    glib, you can easily get yourself to the point where you need a
-    recovery CD - at that point your edit/compile/debug cycle is just
-    too long.
-
- - "sudo make install"
-
-    Now your system is in an undefined state.  You can use e.g. rpm
-    -qV to try to find out what you overwrote, but neither dpkg nor
-    rpm will help clean up any files left over that aren't shipped by
-    the old package.  
-
-    This is most realistic option for people hacking on system
-    components currently, but ostree will be better.
-
- - LXC / containers
-
-   Fails on 3), and 4) is questionable.  Also shares the annoying part
-   of rebuilding distribution packages. LXC is focused on running
-   multiple server systems at the *same time*, which isn't what we
-   want (at least, not right now), and honestly even trying to support
-   that for a graphical desktop would be a lot of tricky work, for
-   example getting two GDM instances not to fight over VT
-   allocations. But some bits of the technology may make sense to use.
-
- - jhbuild + distribution packages
-
-    The state of the art in GNOME - but can only build non-root things -
-    this means you can't build NetworkManager, and thus are permanently
-    stuck on whatever the distro provides.
-
-Who is ostree for?
-------------------
-
-First - operating system developers and testers.  I specifically keep
-a few people in mind - Dan Williams and Eric Anholt, as well as myself
-obviously.  For Eric Anholt, a key use case for him is being able to
-try out the latest gnome-shell, and combine it with his work on Mesa,
-and see how it works/performs - while retaining the ability to roll
-back if one or both breaks.
-
-The rollback concept is absolutely key for shipping anything to
-enthusiasts or knowledable testers.  With a system like this, a tester
-can easily perform a local rollback - something just not well
-supported by dpkg/rpm.  (What about Conary?  See below.)
-
-Also, distributing operating system trees (instead of packages) gives
-us a sane place to perform automated QA **before** we ship it to
-testers.  We should never be wasting these people's time.
-
-Even better, this system would allow testers to *bisect* across
-operating system builds, and do so very efficiently.
-
-The core idea - chroots
------------------------
-
-chroots are the original lightweight "virtualization".  Let's use
-them.  So basically, you install a mainstream distribution (say
-Debian).  It has a root filesystem like this:
-
-               /usr
-               /etc
-               /home
-               ...
-
-Now, what we can do is have a system that installs chroots as a subdirectory
-of the root, like:
-
-               /ostree/gnomeos-3.0-opt-393a4555/{usr,etc,sbin,...}
-               /ostree/gnomeos-3.2-opt-7e9788a2/{usr,etc,sbin,...}
-
-These live in the same root filesystem as your regular distribution
-(Note though, the root partition should be reasonably sized, or
-hopefully you've used just one big partition).
-
-You should be able to boot into one of these roots.  Since ostree
-lives inside a distro created partition, a tricky part here is that we
-need to know how to interact with the installed distribution's grub.
-This is an annoying but tractable problem.
-
-First, we install a kernel+initramfs alongside the distribution's.
-Then, we have a "trampoline" ostree-init binary which is statically
-linked, and boot the kernel with init=/ostree/ostree-init.  This then
-takes care of chrooting and running the init binary.
-
-An important note here is that we bind mount the real /home.  This
-means you have your data.  This also implies we share uid/gid, so
-/etc/passwd will have to be in sync.  Probably what we'll do is have a
-script to pull the data from the "host" OS.
-
-I've decided for now to move /var into /ostree to avoid sharing it
-with the "host" distribution, because in practice we're likely
-to hit incompatibilities.
-
-Do note however /etc lives *inside* the OSTree; it's presently
-versioned and readonly like everything else.
-
-On a pure OSTree system, the filesystem layout will look like this:
-
-               .
-               |-- boot
-               |-- home
-               |-- ostree
-               |   |-- var
-               |   |-- current -> gnomeos-3.2-opt-7e9788a2
-               |   |-- gnomeos-3.0-opt-393a4555
-               |   |   |-- etc
-               |   |   |-- lib
-               |   |   |-- mnt
-               |   |   |-- proc
-               |   |   |-- run
-               |   |   |-- sbin
-               |   |   |-- srv
-               |   |   |-- sys
-               |   |   `-- usr
-               |   `-- gnomeos-3.2-opt-7e9788a2
-               |       |-- etc
-               |       |-- lib
-               |       |-- mnt
-               |       |-- proc
-               |       |-- run
-               |       |-- sbin
-               |       |-- srv
-               |       |-- sys
-               |       `-- usr
-               |-- root
-               
-
-Making this efficient
----------------------
-
-One of the first things you'll probably ask is "but won't that use a
-lot of disk space"?  Indeed, it will, if you just unpack a set of RPMs
-or .debs into each root.
-
-Besides chroots, there's another old Unix idea we can take advantage
-of - hard links.  These allow sharing the underlying data of a file,
-with the tradeoff that changing any one file will change all names
-that point to it.  This mutability means that we have to either:
-
- 1. Make sure everything touching the operating system breaks hard links
-    This is probably tractable over a long period of time, but if anything
-    has a bug, then it corrupts the file effectively.
- 2. Make the core OS read-only, with a well-defined mechanism for mutating
-    under the control of ostree.
-
-I chose 2.
-
-A userspace content-addressed versioning filesystem
----------------------------------------------------
-
-At its very core, that's what ostree is.  Just like git.  If you
-understand git, you know it's not like other revision control systems.
-git is effectively a specialized, userspace filesystem, and that is a
-very powerful idea.
-
-At the core of git is the idea of "content-addressed objects".  For
-background on this, see <http://book.git-scm.com/7_how_git_stores_objects.html>
-
-Why not just use git?  Basically because git is designed mainly for
-source trees - it goes to effort to be sure it's compressing text for
-example, under the assumption that you have a lot of text.  Its
-handling of binaries is very generic and unoptimized.
-
-In contrast, ostree is explicitly designed for binaries, and in
-particular one type of binary - ELF executables (or it will be once we
-start using bsdiff).  
-
-Another big difference versus git is that ostree uses hard links
-between "checkouts" and the repository.  This means each checkout uses
-almost no additional space, and is *extremely* fast to check out.  We
-can do this because again each checkout is designed to be read-only.
-
-So we mentioned above there are:
-
-               /ostree/gnomeos-3.2-opt-7e9788a2
-               /ostree/gnomeos-3.2-opt-393a4555
-
-There is also a "repository" that looks like this:
-
-               /ostree/repo/objects/17/a95e8ca0ba655b09cb68d7288342588e867ee0.file
-               /ostree/repo/objects/17/68625e7ff5a8db77904c77489dc6f07d4afdba.meta
-               /ostree/repo/objects/17/cc01589dd8540d85c0f93f52b708500dbaa5a9.file
-               /ostree/repo/objects/30
-               /ostree/repo/objects/30/6359b3ca7684358a3988afd005013f13c0c533.meta
-               /ostree/repo/objects/30/8f3c03010cedd930b1db756ce659c064f0cd7f.meta
-               /ostree/repo/objects/30/8cf0fd8e63dfff6a5f00ba5a48f3b92fb52de7.file
-               /ostree/repo/objects/30/6cad7f027d69a46bb376044434bbf28d63e88d.file
-
-Each object is either metadata (like a commit or tree), or a hard link
-to a regular file.
-
-Note that also unlike git, the checksum here includes *metadata* such
-as uid, gid, permissions, and extended attributes.  (It does not include
-file access times, since those shouldn't matter for the OS)
-
-This is another important component to allowing the hardlinks.  We
-wouldn't want say all empty files to be shared necessarily.  (Though
-maybe this is wrong, and since the OS is readonly, we can make all
-files owned by root without loss of generality).
-
-However this tradeoff means that we probably need a second index by
-content, so we don't have to redownload the entire OS if permissions
-change =)
-
-Atomic upgrades, rollback
--------------------------
-
-OSTree is designed to atomically swap operating systems - such that
-during an upgrade and reboot process, you either get the full new
-system, or the old one.  There is no "Please don't turn off your
-computer".  We do this by simply using a symbolic link like:
-
-/ostree/current -> /ostree/gnomeos-3.4-opt-e3b0c4429
-
-Where gnomeos-e3b0c4429 has the full regular filesystem tree with usr/
-etc/ directories as above.  To upgrade or rollback (there is no
-difference internally), we simply check out a new tree into
-gnomeos-b90ae4763 for example, then swap the "current" symbolic link,
-then remove the old tree.
-
-But does this mean you have to reboot for OS upgrades?  Very likely,
-yes - and this is no different from RPM/deb or whatever.  They just
-typically lie to you about it =)
-
-A typical model with RPM/deb is to unpack the new files, then use some
-IPC mechanism (SIGHUP, a control binary like /usr/sbin/apachectl) to
-signal the running process to reload.  There are multiple problems
-with this - one is that in the new state, daemon A may depend on the
-updated configuration in daemon B.  This may not be particularly
-common in default configurations, but it's highly likely that that
-some deployments will have e.g. apache talking to a local MySQL
-instance.  So you really want to do is only apply the updated
-configuration when all the files are in place; not after each RPM or
-.deb is installed.
-
-What's even harder is the massive set of race conditions that are
-possible while RPM/deb are in the process of upgrading.  Cron jobs are
-very likely to hit this.  If we want the ability to apply updates to a
-live system, we could first pause execution of non-upgrade userspace
-tasks.  This could be done via SIGSTOP for example.  Then, we can swap
-around the filesystem tree, and then finally attempt to apply updates
-via SIGHUP, and if possible, restart processes.
-
-Configuration Management
-------------------------
-
-By now if you've thought about this problem domain before, you're wondering
-about configuration management.  In other words, if the OS is read only,
-how do I edit /etc/sudoers?
-
-Well, have you ever been a system administrator on a zypper/yum
-system, done an RPM update, which then drops .rpmnew files in your
-/etc/ that you have to go and hunt for with "find" or something, and
-said to yourself, "Wow, this system is awesome!!!" ?  Right, that's
-what I thought.
-
-Configuration (and systems) management is a tricky problem, and I
-certainly don't have a magic bullet.  However, one large conceptual
-improvement I think is defaulting to "rebase" versus "merge".
-
-This means that we won't permit direct modification of /etc - instead,
-you HAVE to write a script which accomplishes your goals.  To generate
-a tree, we check out a new copy, then run your script on top.
-
-If the script fails, we can roll back the update, or drop to a shell
-if interactive.
-
-However, we also need to consider cases where the administrator
-modifies state indirectly by a program.  Think "adduser" for example.
-
-Possible approaches:
-
- 1. Patch all of these programs to know how to write to the writable
-    location, instead of the R/O bind mount overlay.
- 2. Move the data to /var
-
-What about "packages"?
-----------------------
-
-There are several complex and separate issues hiding in this seemingly
-simple question. 
-
-I think OSTree always makes sense to use as a core operating system
-builder and updater.  By "core" here I mean the parts that aren't
-removable.  Debian has Essential: yes, any other distribution has this
-too implicitly in the set of dependencies for their updater tool.
-
-Now, let me just say I will absolutely support using something like
-apt/yum/zypper (and consequently deb/rpm) on top of OSTree.  This
-isn't trivial, but there aren't any conceptual issues.
-
-Concretely for example, RPM or .deb might make sense as a delivery
-vehicle for third party OS extensions.  A canoncial example is the
-NVidia graphics driver.
-
-If one is using OSTree to build an *operating system*, then there has
-to be some API for applications.  And that demands its own targeted
-solution - something like an evolved glick (zeroinstall is also
-similar).
-
-Current package systems are totally broken for application deployment
-though; for example, they will remove files away from under running
-applications on update.  And we clearly need the ability to install
-and upgrade applications without rebooting the OS.
-
-Details of RPM installation
----------------------------
-
-We should be able to install LSB rpms.  This implies providing "rpm".
-The tricky part here is since the OS itself is not assembled via RPMs,
-we need to fake up a database of "provides" as if we were.  Even
-harder would be maintaining binary compatibilty with any arbitrary
-%post scripts that may be run.
-
-What about BTRFS?  Doesn't it solve everything?
------------------------------------------------
-
-In short, BTRFS is not a magic bullet, but yes - it helps
-significantly.  The obvious thing to do is layer BTRFS under dpkg/rpm,
-and have a separate subvolume for /home so rollbacks don't lose your
-data.  See e.g.
-<http://fedoraproject.org/wiki/Features/SystemRollbackWithBtrfs>
-
-As a general rule an issue with the BTRFS is that it can't roll back
-just changes to things installed by RPM (i.e. what's in rpm -qal).
-
-For example, it's possible to e.g. run yum update, then edit something
-in /etc, reboot and notice things are broken, then roll back and have
-silently lost your changes to /etc.
-
-Another example is adding a new binary in /usr/local.  You could say,
-"OK, we'll use subvolumes for those!".  But then what about /var (and
-your VM images that live in /var/lib/libvirt ?)
-
-Finally, probably the biggest disadvantage of the rpm/dpkg + BTRFS
-approach is it doesn't solve the race conditions that happen when
-unpacking packages into the live system.  This problem is really
-important to me.
-
-Note though ostree can definitely take advantage of BTRFS features!
-In particular, we could use "reflink"
-<http://lwn.net/Articles/331808/> instead of hard links, and avoid
-having the object store corrupted if somehow the files are modified
-directly.
-
-Other systems
--------------
-
-I've spent a long time thinking about this problem, and here are some
-of the other possible solutions out there I looked at, and why I
-didn't use them:
-
- - Git: <http://git-scm.com/>
-
-    Really awesome, and the core inspiration here.  But like I mentioned
-    above, not at all designed for binaries - we can make different tradeoffs.
-
- - bup: <https://github.com/apenwarr/bup>
-
-    bup is cool.  But it shares the negative tradeoffs with git, though it
-    does add positives of its own.  It also inspired me.
-
- - git-annex: <http://git-annex.branchable.com/git-annex/>
-
-   Looks interesting; I think this will share the same negative tradeoffs with git
-   as far as using it for an OS goes.
-
- - schroot: <http://www.debian-administration.org/articles/566>
-
-   Like LXC/containers, but just using a chroot.
-
- - NixOS: <http://nixos.org/>
-
-    The NixOS people have a lot of really good ideas, and they've definitely
-    thought about the problem space.  However, their approach of checksumming
-    all inputs to a package is pretty wacky.  I don't see the point, and moreover
-    it uses gobs of disk space.
-
- - Conary: <http://wiki.rpath.com/wiki/Conary:Updates_and_Rollbacks>
-
-   If rpm/dpkg are like CVS, Conary is closer to Subversion.  It's not
-   bad, but ostree is better than it for the exact same reasons git
-   is better than Subversion.
-
- - BTRFS: <http://en.wikipedia.org/wiki/Btrfs>
-
-   See above.
-
- - Solaris IPS: <http://hub.opensolaris.org/bin/view/Project+pkg/>
-
-   Rollback is ZFS level, so I think this shares the same tradeoffs as BTRFS+RPM/deb.
-   They probably have some vertical integration though which definitely helps.
-   Obviously we can't use ZFS.
-
- - Jhbuild: <https://live.gnome.org/Jhbuild>
-
-   What we've been using in GNOME, and has the essential property of allowing you
-   to "fall back" to a stable system.  But ostree will blow it out of the water.
-
-Development
------------
-
- - OSTree wiki page: <https://live.gnome.org/OSTree>
-
- - ostbuild wiki page: <https://live.gnome.org/OSTree/Ostbuild>
-
- - Git repository: <http://git.gnome.org/browse/ostree/>
-
- - Deploying OSTree in the Gnome servers: <https://bugzilla.gnome.org/show_bug.cgi?id=669772>
+https://live.gnome.org/Projects/OSTree


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]