Re: Adapting an existing distribution to OSTree



Hi Gatis,

On 24.06.2016 11:28, Gatis Paeglis wrote:

> My next step is to convert AGL to OSTree based system.


You can see an example of how I did this in:


git clone https://codereview.qt-project.org/qt/qtotaupdate


with the online documentation available in: http://doc.qt.io/QtOTA/


The above script converts Yocto/OpenEmbedded based distribution* into OSTree based image.

* Boot-2-Qt distribution: git clone https://codereview.qt-project.org/yocto/meta-boot2qt


I have not tried it with AGL, but it could be adapted with some minor modifications, probably. Patches are welcomes of course ��


Thank you for sharing these valuable links and resources. This week I am away on a trip to Hong Kong Open Source conference. I already had a quick look at convert_to_ostree_sysroot and commit_generated_tree from script qt-ostree. I need something similar for AGL. As soon as I get back home I will explore the script in more details.

Thanks, Leon


Gatis Paeglis.


From: ostree-list <ostree-list-bounces gnome org> on behalf of Leon Anavi <leon anavi konsulko com>
Sent: Wednesday, June 15, 2016 6:19:02 PM
To: Colin Walters; ostree-list gnome org
Subject: Re: Adapting an existing distribution to OSTree
 
Hi Colin,

On 15.06.2016 16:18, Colin Walters wrote:
> Hi,
>
> On Wed, Jun 15, 2016, at 05:46 AM, Leon Anavi wrote:
>> Hi,
>>
>> I am a working on the integration of OSTree with GENIVI SOTA project in
>> Automotive Grade Linux (AGL).
> Great!  I'm really interested in helping this type of thing succeed.

Thank you for your prompt reply and the provided additional information.
I hope that we will be able to move this project forward.

>
>> The client downloads the
>> binaries from the server, and then hands them over to the system for
>> installation.
> In OSTree these phases are intentionally mixed - the client downloads
> updates incrementally *unpacked* into the repo, without affecting the current system,
> and "deployment" is a second phase which also doesn't affect the running
> system other than to make the hardlink farm and a new bootloader entry.
>
> I see this as an advantage over e.g. yum/apt-get and other package
> managers which stage the compressed packages and then unpack them
> only after all are downloaded, leading to a transient space usage spike.
>
> It's also an advantage over dual partition schemes because it avoids
> doubling the OS footprint.
>
>> After reviewing several open source solutions for
>> installation strategies during the AGL face to face meeting in May the
>> interested parties decided to try OSTree. More technical details about
>> the discussions are available in AGL wiki and mailing list:
>> https://wiki.automotivelinux.org/agl-distro/may2016-f2f
>> https://lists.linuxfoundation.org/pipermail/automotive-discussions/2016-May/002077.html
> I'm not finding a lot of details on the comparison there but I would
> point out:
> https://ostree.readthedocs.io/en/latest/manual/related-projects/
>
> I'd be happy to accept links to other projects - I was unaware of mender.io
> for example.
>
>> AGL is Yocto/OE based distribution. I have already integrated RVI SOTA
>> client which is written in the Rust programming language to the upstream
>> of AGL.
> Neat.  I would love to be able to use more Rust but I think for various
> conservative reasons the ostree stack is going to remain C for quite a
> while.  First I'd like to move more in the direction of using Linux-specific features/syscalls,
> which is certainly possible with rust but then I'm fighting `libstd` in the
> same way we're avoiding `GFile` mostly now,
> and second hard requiring Rust would close out some more conservative targets
> right now.
>
> That said I'd be fine having whatever we need to enable Rust bindings to
> libostree in the core.  I think a lot of that hopefully comes for free from
> https://github.com/gi-rust ?

I have already added Yocto/OE recipe for building the client in AGL
using layer meta-rust. I believe that if in long term Rust is
contentious for the SOTA client, ATS Advanced Telematic Systems GmbH,
the company working on GENIVI SOTA project, may be interested in
rewriting it in C.

I am trying to make progress step by step. Therefore my first step was
to create the Yocto/OE recipes for building up to date version of OSTree
in AGL. The second step was to get familiar with OSTRee by trying it out
with AGL running on Intel NUC. I successfully initialized OSTree
repository for a directory with text files, added a few commits,
synchronized it with another repository on the same device with ostree
pull-local, synchronized with a repository on another machine using
ostree pull. I have also generated static-delta from one repository and
applied them offline to another. My next step is to convert AGL to
OSTree based system.

>
>> Now I am trying to adapt AGL as an OSTree-enabled system and to generate
>> OSTree commits from the Yocto builds. I have already read chapter
>> "Adapting existing mainstream distributions" from the official
>> documentation, I tried out GNOME continuous in a virtual machine and I
>> looked for hints from the bash scripts in directory tests. I still need
>> more information.
> GContinuous is more of a demo.  My employer uses OSTree indirectly
> via rpm-ostree which in turn derives content from production
> distributions like CentOS and Fedora.  See http://www.projectatomic.io/download/
> for links to VMs and ISO installers.
>
> But that said the distinction between ostree and higher level tools
> like rpm-ostree is precisely to enable sharing code with other
> distributions.
>
>> Could you please provide me more practical guidelines and examples for
>> converting an image built with bitbake to OSTree based image?
> The quick answer is I can't justify a lot of time on Yocto right now, but I
> can help with questions.  I already have to spend a lot of time integrating
> with the Fedora/CentOS ecosystems.
>
> If you want to try building custom content with a CentOS base using
> rpm-ostree, see:
> https://github.com/projectatomic/rpm-ostree/blob/master/docs/manual/compose-server.md
>
> More details on Yocto:
>
> What GContinuous does is take the target root dir from bitbake,
> tars it up, then imports it again into ostree.  See:
> https://git.gnome.org/browse/gnome-continuous-yocto/tree/meta-gnomeos/classes/gnomeos-contents.bbclass?h=gnomeostree-3.14-dizzy
> and then commits it into ostree:
> https://git.gnome.org/browse/gnome-continuous/tree/src/js/tasks/task-build.js#n1173

I am looking at the provided links. Right now I want to manually adapt
AGL to OSTree-based system. Following the recommendations from the
documentation I have moved the directories and I added rules for
systemd-tmpfiles.

What are the exact steps to manually initialize OSTree repository for
the whole system, add a commit and create a new deployment?

I have tried by executing the following commands directly on the
embedded device:
ostree admin os-init testos
ostree commit -b testos/foo/bar --tree=ref=testos/foo/bar
--subject="Initial commit" --body="commit 1"
ostree admin deploy testos/foo/bar –os=testos

But I got the following error:
error: openat: No such file or directory

Best regards, Leon

> It's quite slow, but the goal of that system is to use Yocto only for a slow-changing
> base, then have fast-moving content on top using a highly custom build
> system that uses git+ostree.
>
> This could certainly be improved and ideally upstreamed into Yocto core.
> To really make OSTree fast, one needs to do things like GContinuous
> is doing where each package/component is stored as a separate branch,
> and then unioned to compute the final tree.  We're increasingly doing
> more things like this in rpm-ostree.   It basically needs a lot
> of integration into the buildsystem.
>
>  
>
> _______________________________________________
> ostree-list mailing list
> ostree-list gnome org
> https://mail.gnome.org/mailman/listinfo/ostree-list

--
Leon Anavi
Software Engineer
konsulko.com

_______________________________________________
ostree-list mailing list
ostree-list gnome org
https://mail.gnome.org/mailman/listinfo/ostree-list

-- 
Leon Anavi
Software Engineer
konsulko.com


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]