Re: How is OSTree designed to be safe from power loss damage?

[Leon Woestenberg <leon sidebranch com>]

Hi Phil,

I have looked through the existing work (excellent) and in general OpenEmbedded/Yocto need a bit more flexibility regarding the policy the implementation implies.

- I think we should also support OpenEmbedded (which is distro-less) besides Yocto (which always comes with Poky distro). I think what I saw depends on Yocto currently

-  the "/usr merge" logic might be put into a seperate Bitbake class, although the existing discussions on /usr merge ended without much conclusion yet (but OSTree might be an actual use-case interesting for many dev's). The discussion needs to be re-opened, I can do that.

- The current implementation enforces that a first-boot (initial / live) image pulls the OSTree repo during the image creating build-host-side. When I was creating my OpenEmbedded / OSTree system  before I learned about meta-updater, I had this logic on the target-side. Basically, my target would be initialized without a deployment, but enough tools to pull and deploy the first OSTree. This is really more of a production work flow difference, I have no strong preference for one or the other, but maybe we should support thin initial provisioning like this, because maybe it is yet unclear what OSTree "OS" the platform will be provisioned with in the first place.

- make sure we are strongly layered, I would like to see a solution that works with ostree only, although I do understand the need for higher-level server/clients, the ostree specific layer should not depend on higher levels.

Not sure about other topics at this moment.

Regards,

Leon.

 

On Mon, Jan 9, 2017 at 3:52 PM, Phil Wise <phil advancedtelematic com> wrote:

Hi Leon,

On 09.01.2017 15:29, Leon Woestenberg wrote:
>
> +CC: Leon Anavi and Anton Gerasimov.
>
> I think discussions will happen here on the OSTree mailing list or
> OpenEmbedded (or Yocto) mailing list, depending on the core of the subject.
>
> The existing work is AGL related, this is where I saw Leon and Anton do
> what I was trying to do. :)

meta-updater is independent of AGL now, so it should work with any
OE-based project.  We've put together a simple example using poky and
the R-Pi here:

https://github.com/advancedtelematic/garage-quickstart-rpi

I'd be keen to get OSTree support upstreamed into Yocto/OE: is there
anything we can do to help that process?


Cheers,

Phil


> On Mon, Jan 9, 2017 at 3:12 PM, Colin Walters <walters verbum org
> <mailto:walters verbum org>> wrote:
>
>     __
>
>     On Mon, Jan 9, 2017, at 07:37 AM, Leon Woestenberg wrote:
>>     I would like to make OSTree a first class citizen in upstream
>>     Yocto/OpenEmbedded, a lot has been done already by Leon Anavi and
>>     Anton Gerasimov in meta-updater.
>>     (https://github.com/advancedtelematic/meta-updater
>>     <https://github.com/advancedtelematic/meta-updater>)

>
>     Awesome, happy to help with anything I can here.
>
>>
>>     The question was not so much about the atomic swap, but more about
>>     what happens if the system was power-cycled or reset in the middle
>>     of pulling objects from a remote, or in the middle of deploying an OS?
>>     In other words, what happens if I reset the device halfway
>>     creating the hardlinks, on the next boot cycle?
>
>     Okay, so there are two phases here.  Downloading into the repo, and
>     creating deployments.
>
>     For downloads, the latest on this is:
>     https://github.com/ostreedev/ostree/commit/27a45e2edbd1c4551b801a957cafcb27b3417fd3
>     <https://github.com/ostreedev/ostree/commit/27a45e2edbd1c4551b801a957cafcb27b3417fd3>
>     Specifically:
>     https://github.com/ostreedev/ostree/commit/27a45e2edbd1c4551b801a957cafcb27b3417fd3#diff-23d4c8887c7dda91613c9451bf3fe7c0R1281
>     <https://github.com/ostreedev/ostree/commit/27a45e2edbd1c4551b801a957cafcb27b3417fd3#diff-23d4c8887c7dda91613c9451bf3fe7c0R1281>
>
>     Basically, pending objects (files) are staged in a directory per
>     kernel "boot id", which is
>     unique per boot.  Hence, if the power is lost any pending objects
>     will be discarded (
>     we currently do a time-based pruning of the tmpdir).
>     This is a bit conservative obviously - we could *try* resuming from
>     them and if the checksum doesn't match, redownload, but so far it
>     hasn't gone up in priority.
>
>     For deployments, it's a lot simpler - we're just making a hardlink
>     farm and copying
>     forward changes in /etc.  If the system crashes before, on the next
>     upgrade attempt,
>     we'll just garbage collect the pending hardlink farm and redo it.
>
>
>
>
>

> _______________________________________________
> ostree-list mailing list
> ostree-list gnome org
> https://mail.gnome.org/mailman/listinfo/ostree-list
>

--
Phil Wise, ATS Advanced Telematic Systems GmbH
Kantstrasse 162, 10623 Berlin
Managing Directors: Dirk Pöschl, Armin G. Schmidt
Register Court: HRB 151501 B, Amtsgericht Charlottenburg
_______________________________________________
ostree-list mailing list
ostree-list gnome org
https://mail.gnome.org/mailman/listinfo/ostree-list